Chemistry society lifts well-written papers from 58% to 81%

Challenge

Why the society needed AI tuned to its content

As the society grew, it needed more production capacity, but broad commercial AI tools did not perform well on its content.

The society, a leading home for chemistry professionals and a major scientific publisher, publishes over 80 journals. Its Production Operations team handles technical editing and proof review and, as the organization grew, now relies on a global supply chain. The team also develops new technologies, particularly AI.

The team explored programmatic language modification but found commercial tools were “really striking out” on its content. It needed suggestions for academic language usage, such as sentence structures and discipline-specific vocabulary.

Many of the services we offer to publishers today are the fruits of our collaboration with this society. It is together that we have identified challenges, thought of solutions, and developed and vetted new technologies.

Juan Castro

CEO and Co-founder, Writefull

Solution

How the society rolled out Writefull across its production workflow

After a competitive RFP, the society chose Writefull for models tailored to scientific content, and has since adopted four of its tools.

Writefull’s publisher offering supports each stage of the pipeline, from submission to copy editing to quality assurance. The two teams began by co-developing services that the society has since operationalized, with Writefull’s models tailored to a niche of scientific content.

1. Score language after copyediting
   A tailored Manuscript Categorization API scores language quality after copyediting and was benchmarked against the society’s analysts.
2. Check metadata automatically
   The Metadata API extracts manuscript elements, such as author names and affiliations, for post-editing quality checks.
3. Pre-edit manuscripts with LEA
   Language Edit Assist (LEA) auto-applies a highly customized subset of Writefull’s edits before copyediting begins.
4. Refine edits with internal data
   Editors evaluated 35,000 LEA edits; after refinement on internal data, 99.04% were applied without error.

Who was involved: The society’s Production Operations team, quality analysts and copyeditors, and Writefull, including CEO and co-founder Juan Castro.

Results

What the society achieved with Writefull’s APIs

Writefull’s services delivered time and cost savings, with LEA raising the share of well-written papers from 58% to 81% in a 715-paper comparison.

• 2 workflows compared on similar-quality manuscripts, one with LEA applied before technical editing and one without.
• 23 percentage points more papers rated well-written with LEA, versus the group without it.
• 0.96% of 35,000 reviewed LEA edits introduced errors, after refinement using the society’s own data.
• 4 Writefull tools adopted over the years across quality review and copyediting preparation.

What it means: With more papers arriving at copyediting in good shape, copyeditors can focus on high-value edits rather than basic grammar fixes such as articles and punctuation. Programmatic checks also give the society quality data it could not collect manually, and new insights into how it operates.

Measurement note: Society’s Production Operations team analysis; 715 papers randomly split into two workflows.

Outlook

What comes next for the society and Writefull

The society and Writefull continue to refine their processes and are assessing how adoption affects throughput and quality, with staff buy-in growing as refinements improve tool performance.

FAQs

How the society uses Writefull day to day