188bet½ð±¦²©¹ÙÍøµÇ¼

How One Chemist Playing Hooky From His Day Job Helped To Get A Paper Retracted From Chemistry’s Flagship Journal

[In the vein of the recent Chemical & Engineering News perspective on “�, [see here for illuminating articles on , and ] it seems an opportune moment to revisit a recent example of how the Internet augmented the self-correcting nature of the scientific process, and helped to boot a poorly researched paper out of chemistry’s flagship journal before it hit publication.

Why publish this on a blog for sophomore organic chemistry students?ÌýReproducibility of scientific experiments is in the news [] []. ÌýIt’s a great lesson for non-experts in how organic chemistry experiments are conducted… and a reminder that while bad papers sometimes sneak through, the organic chemistry literature has a lot of integrity.]

On July 21, 2009 a veryâ€� interesting article was published in the ASAP (“As Soon As Publishable”) section of the Journal of The American Chemical Society (JACS). The authors claimed that treating the following molecule (4-chlorobenzyhydrol) A with the base sodium hydride (NaH) resulted in the formation of the product B (4-chloroacetophenone) in 85% yield. The paper had several other very similar examples.

If you can read structural formulae, you can see that what’s happened here is that C-H had broken and we’ve formed a new C-O double bond (“Ï€ bond”) – a reaction known as ´Ç³æ¾±»å²¹³Ù¾±´Ç²Ô.Ìý

The authors claimed that sodium hydride was acting as the oxidant in this reaction.

The response of many chemists was to furrow an eyebrow and say… “huh?”.ÌýSince the very definition of oxidation is toÌýbreak C-H bonds, and NaH is a reagent forÌýforming bonds to H, it seems highly curious (to say the least) that this reagent might act as anÌýoxidant.

In particular the authors were vague about how NaH could perform this task (a process that organic chemists refer to as the “reaction mechanism”), and even said that sodium metal (which should have produced the same anion as NaH) did not promote oxidation. In retrospect, that was a crucial clue that something else was going on.

ChapterÌý1: The Thread Hijack

Meanwhile, in one of the dark corners of the Internet where chemists hung out at the time, the blog , author Paul Docherty had just finished anÌýÌýabout a recent synthesis of Tricholomalides A and B by Sam Danishefsky’s lab at Sloan-Kettering.

The second comment in the thread took the discussion in a completely different direction, pointing to the “NaH as oxidant” paper.

ChapterÌý2: Liveblogging The Reaction

Docherty’s curiosity was piqued. Noting the simplicity of the example, and realizing that these same starting materials were present in his own laboratory, the next day (July 22)Ìýhe took some time off whatever he was supposed to be doing at his day job at Arrow Pharmaceuticals to check this reaction out for himself.

At 10:40 a.m.Ìýthe NaH was added to the alcoholÌýunder an atmosphere of nitrogen at 0°C and stirred.
“Cue fizzing”, said Docherty.

12:10 p.m.Ìý“‘Tis Orange!”ÌýAfter warming to room temperature, the reaction had turned orange [color is a parts-per-million phenomenon, so this can often be a misleading indication for reaction progress].
1:20 p.m. First AnalysisÌýThe reaction was Ìýanalyzed by liquid-chromatography mass spectroscopy (LCMS), which showed both starting material and product to be present.

ChapterÌý3: The Result

The next day, Docherty took an NMR of the product mixture (a tool chemists use to determine the structure and purity of organic compounds) and was able to determine that the reaction proceeded in 15% yield – a far cry from the 85% yieldÌýreported in the paper. [The NMR shows strong peaks corresponding to starting material at 7.5 and 2.0 ppm; the remainder of the peaks are product]

ChapterÌý4:ÌýThe Community Weighs In

While Docherty was live-blogging his reaction, others attempted to reproduce the paper’s results.

One commenterÌýfound that the reaction did not proceed at allÌýin the absence of oxygen, which seemed to bear out the general sentiment that NaH was not in fact the oxidant in this reaction.

However, when oxygen was added, the same reaction went to completion, giving the desired product:

Others reported similar observations.

Note the timestamp on these posts.ÌýThese results were obtained within 72 hours of the paper appearing online.Ìý

Interestingly, one of the commenters referenced a which showed that there was precedent for NaH (usually used as a suspension in mineral oil) oxidizing alcohols to carboxylic acids in the presence of air. In other words, the authors of the 2009 paper didn’t do their homework.

Step 5: “This Manuscript Has Been Withdrawn For Scientific Reasons”

Docherty’s results, as well as those of theÌýcommenters, suggested that the true “oxidant” in the reaction was not NaH as the authors of the paper claimed, but molecular oxygen.

After this, things quieted down. ÌýThere was no direct contact with the lab of the PI responsible for publishing the paper. However, a short period of time later, the following noteÌýappeared where the link to the paper formerly appeared.

“This manuscript has been withdrawn for scientific reasons”

Conclusion:Ìý

What can we learn from this? Organic chemistry experiments have have several features which favor a high degree of reproducibility.

• Large sample size. Running a reaction on a millimole (1/1000 of a mole) of material still means you’re sampling 6 x 10²⁰ molecules [Docherty ran his experiment on 4 millimoles of material]. Compare this to psychology studies, which might be done on a few dozen individuals at the lower bound. Safe to say that no chemist has ever had to manipulate a p-value.
• Reasonable cost of most reagents (chemicals are generally cheapÌýwhen compared with enzymes, antibodies, DNA sequences, etc)
• Short time frame – most experiments are complete within 24 h. The example above was replicated within 72h of the paper appearing online. This also compares favourably with biomedical studies, which can take months or even years to conduct (if you’re waiting for an organism with a knockout gene to reach a certain developmental stage, for instance)
• Widely held expertise – With no offence meant to Docherty, there was nothing particularly “special” or unique about his training in organic synthesis. He ran a reaction that thousands of other people in the worldÌýcould have run just as expertly. What put him in a unique position was that he ran a blog read by thousands of other chemists, so his results were widely distributed. Contrast this to molecular biology, where success in reproducing an experiment might require considerable practice in conducting a particular experimental technique.
  “You can’t give me and Julia Child the same recipe and expect an equally good meal”, said Jeff Settleman of Calico Life Sciences (). When “Julia Child effects” arise in organic chemistry, there’s likely a good reason for it (see below).

• Minimal variables – Ìýfor organic chemistry, controlling Ìýthe temperature, Ìýpurity of reagents, purity of solvent, and atmosphere is usually enough.Ìý Traces of water or oxygen can cause problems, especially on small scale, but these can generally be dealt with. Controlling the variables in biomedical papers, by contrastcan be a nightmare. Experiments are notoriously sensitive to small differences in protein sources, to contamination, or other weird factors (Ìýfor example). ÌýFor some examples on how easy it can be to mess up, here are someÌý. Let’s not even mention controlling variables in psychology. One researcher, being told that her results were much weaker upon replication, said that the reason was that the replicated study used American college students, while hers used Italians.

Given these, we can safely say that the integrity of results in the organic chemistry literature is high.ÌýIf problematic research papers are found, attention will likely be drawn to them, and given the speed at which their results can be falsified, their half-life in the literature will be short. The Internet has increased this half-life dramatically.Ìý

2013 Redux

If you look through the on the article at Totally Synthetic, you might noteÌýseveral grumblings about another reaction organic chemists found to be capricious – the . ÌýIn an interesting postscript to this story, in 2013 this reactionÌýwas tested by the anonymous authors of the websiteÌý. ÌýThe authors also had, or as they put it, “it requires mojo“.

However, after discussions with one of the paper’s primary authors (who had one of his graduate students reproduce the work, in epicÌýdetail, with accompanying pictures), they were able to coax the reaction to produce the desired product.

A sharp eyed anonymous commenter pointed out that the “successful” runs of this reaction had greater trace water impurities in the IBX than “unsuccessful” runs, leading the commenter to suggest adding a small amount of H₂O to the reaction as an additive. And lo! , leading the Blog Syn authors to suggest that the H₂O adduct of IBX, not IBX itself, was in fact the active oxidant.

I include this because it’s a great example of how “Julia Child Effects” (i.e. two people given the same recipe produce results of differing quality) arise for sound scientific reasons, and further exploration of variables can lead to a greater understanding of how reactions work.

The wholeÌýarticle might be one of the best chemistry blog posts ever published,ÌýÌýfor sheer detail. .

On a final note, best wishes to Paul, wherever you may be, for everything you did in building Totally Synthetic. It is dearly missed.