{"id":106350,"date":"2021-10-27T07:00:13","date_gmt":"2021-10-27T11:00:13","guid":{"rendered":"http:\/\/kendallharmon.net\/?p=106350"},"modified":"2021-10-27T17:33:04","modified_gmt":"2021-10-27T21:33:04","slug":"nature-giant-free-index-to-worlds-research-papers-released-online","status":"publish","type":"post","link":"https:\/\/kendallharmon.net\/?p=106350","title":{"rendered":"(Nature) Giant, free index to world\u2019s research papers released online"},"content":{"rendered":"

In a project that could unlock the world\u2019s research papers for easier computerized analysis, an American technologist has released online a gigantic index of the words and short phrases<\/a> contained in more than 100 million journal articles \u2014 including many paywalled papers.<\/p>\n

The catalogue, which was released on 7 October and is free to use, holds tables of more than 355 billion words and sentence fragments listed next to the articles in which they appear. It is an effort to help scientists use software to glean insights from published work even if they have no legal access to the underlying papers, says its creator, Carl Malamud. He released the files under the auspices of Public Resource, a non-profit corporation in Sebastopol, California that he founded.<\/p>\n

Malamud says that because his index doesn\u2019t contain the full text of articles, but only sentence snippets up to five words long, releasing it does not breach publishers’ copyright restrictions on the re-use of paywalled articles. However, one legal expert says that publishers might question the legality of how Malamud created the index in the first place.<\/p>\n

Some researchers who have had early access to the index say it\u2019s a major development in helping them to search the literature with software \u2014 a procedure known as text mining. Gitanjali Yadav, a computational biologist at the University of Cambridge, UK, who studies volatile organic compounds emitted by plants, says she aims to comb through Malamud\u2019s index to produce analyses of the plant chemicals described in the world\u2019s research papers. \u201cThere is no way for me \u2014 or anyone else \u2014 to experimentally analyse or measure the chemical fingerprint of each and every plant species on Earth. Much of the information we seek already exists, in published literature,\u201d she says. But researchers are restricted by lack of access to many papers, Yadav adds.<\/p>\n

Read it all<\/a>.<\/p>\n

\n

A big development for text-mining? @carlmalamud<\/a> has released online a giant, free index of 355 billion words and sentences in 107 million science papers. By @hollyelse<\/a> in @nature<\/a> https:\/\/t.co\/FDE1mlfwbg<\/a><\/p>\n

— Richard Van Noorden (@Richvn) October 26, 2021<\/a><\/p><\/blockquote>\n