Free Republic
Browse · Search
News/Activism
Topics · Post Article

Skip to comments.

Search LibGen, the Pirated-Books Database That Meta Used to Train AI
The Atlantic ^ | 3/20/2025 | Alex Reisner

Posted on 03/20/2025 3:15:59 PM PDT by nickcarraway

Millions of books and scientific papers are captured in the collection’s current iteration.

Editor’s note: This search tool is part of The Atlantic’s investigation into the Library Genesis data set. You can read an analysis about LibGen and its contents here. Find The Atlantic’s search tool for movie and television writing used to train AI here.

Disclaimer: LibGen contains errors. You may, for example, find books that list incorrect authors. This search tool is meant to reflect material that could be used to train AI programs, and that includes material containing mistakes and inaccuracies.

(Excerpt) Read more at theatlantic.com ...


TOPICS: Business/Economy; Constitution/Conservatism; Culture/Society; News/Current Events
KEYWORDS: ai; archives; libraries; machinelearning; software

1 posted on 03/20/2025 3:15:59 PM PDT by nickcarraway
[ Post Reply | Private Reply | View Replies]

To: nickcarraway

I use Libgen from time to time, searching for out of print books. Sometimes it’s there, sometimes it’s not.


2 posted on 03/20/2025 3:20:25 PM PDT by jimtorr
[ Post Reply | Private Reply | To 1 | View Replies]

To: jimtorr
I use Anna’s Archieve: 42 million books, 98 million papers

https://annas-archive.se/

3 posted on 03/20/2025 3:33:47 PM PDT by thegagline (Sic semper tyrannis! Trump & Vance, 2024! (Formerly) Goldwater & Thomas Sowell)
[ Post Reply | Private Reply | To 2 | View Replies]

To: nickcarraway

It’s simple, I can save you a LOT of time. Garbage in, garbage out.


4 posted on 03/20/2025 3:37:57 PM PDT by Shady (The Force of Liberty must prevail for the sake of our Children and Grandchildren...)
[ Post Reply | Private Reply | To 1 | View Replies]

To: thegagline

i tried usign that and downloading a book- but it kept timing out


5 posted on 03/20/2025 4:52:01 PM PDT by Bob434 (Time flies like an arrow, fruit flies like a banana)
[ Post Reply | Private Reply | To 3 | View Replies]

To: nickcarraway
This search tool is meant to reflect material that could be used to train AI programs, and that includes material containing mistakes and inaccuracies.

This is kind of like the situation with modern bibles that are translated from corrupted manuscripts.

Why don't they just turn an AI LLM loose on the internet to train it? That would provide interesting results.

6 posted on 03/20/2025 8:07:52 PM PDT by Disambiguator
[ Post Reply | Private Reply | To 1 | View Replies]

To: Bob434

If you go to,the lower right hand corner you will see alternate links. The link that I provided works best for me. So far I’ve downloaded a couple of thousand books. Currently, I pay for faster download speeds. The process is easy: buy an Amazon gift card and pay them. You will get a key number which will five you access. I am currently downloading Penguin Classes. This site gives me the ability to read books on my ipad which is perfect for me as reading books in paper form is difficult due to some visual limitations.


7 posted on 03/20/2025 9:11:58 PM PDT by thegagline (Sic semper tyrannis! Trump & Vance, 2024! (Formerly) Goldwater & Thomas Sowell)
[ Post Reply | Private Reply | To 5 | View Replies]

To: thegagline

Thank you for the reply. I will give that a try. I read on a galaxy tablet which h I, ike too. It’s about the size of an iPad, maybe a bit larger. 12x9 I think which is perfect size. I Wil. Check out their selections- my relatives like to read chrsitian fiction novels- Amazon has a bunch, but it gets ,ind of expensive. I ,ike christian Sci fi myself. Older books doesn’t bother us- zwe don’t need the latest and newest.


8 posted on 03/20/2025 10:11:57 PM PDT by Bob434 (Time flies like an arrow, fruit flies like a banana)
[ Post Reply | Private Reply | To 7 | View Replies]

To: thegagline

Sorry to bother you again but what do you use for your reader app? I use kindle, but not sure how to get the book onto kindle?


9 posted on 03/21/2025 11:06:05 PM PDT by Bob434 (Time flies like an arrow, fruit flies like a banana)
[ Post Reply | Private Reply | To 7 | View Replies]

To: thegagline

Ok, I think I have how to read on kindle- the site has instructions- I just have to convert to kindle to read it. Trying to do so now.


10 posted on 03/21/2025 11:10:13 PM PDT by Bob434 (Time flies like an arrow, fruit flies like a banana)
[ Post Reply | Private Reply | To 7 | View Replies]

To: thegagline

Sorry, one last post to tell you I got it to the kindle- and tried the a,turnstile site for download and it was much faster. A few seconds ds for a 0.24 mb file- a mystery novel- just wanted to say thank you for posting about the site. Didn’t know it existed- their library is huge!


11 posted on 03/21/2025 11:53:10 PM PDT by Bob434 (Time flies like an arrow, fruit flies like a banana)
[ Post Reply | Private Reply | To 7 | View Replies]

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
News/Activism
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson