Show HN: ArXiv-txt, LLM-friendly ArXiv papers

 (arxiv-txt.org)

22 points | by jerpint 142 days ago

6 comments

  • westurner 141 days ago
    If you train an LLM on only formally verified code, it should not be expected to generate formally verified code.

    Similarly, if you train an LLM on only published ScholarlyArticles ['s abstracts], it should not be expected to generate publishable or true text.

    Traceability for Retraction would be necessary to prevent lossy feedback.

  • owalerys 135 days ago
    Really clean API design, I'm a fan!
  • lgas 141 days ago
    It just extracts the abstracts?
    [-]
    • jerpint 141 days ago
      For now , yes - abstracts and other metadata
      [-]
      • rrekaf 141 days ago
        do you plan on adding descriptions of figures and tables?
        [-]
        • jerpint 141 days ago
          will probably focus on getting the text out of the papers first, figures might be a good next step after that
  • sbpost 141 days ago
    The example you give doesn't seem to work - the raw txt does not have authors.
    [-]
    • jerpint 141 days ago
      you're right - I hadn't noticed! I fixed it now, thanks for pointing it out
  • jmartin2683 141 days ago
    This would be awesome wrapped in an MCP server/tool call :)
    [-]
    • jerpint 141 days ago
      whoa - i haven't yet played with MCP - might be a good first project!
  • cchance 140 days ago
    Was super excited that it was going to be the actual papers, kinda cool but just being abstracts doesn't go very far, good luck getting the papers working thats gonna be pretty cool once working, then to feed it all into a vector db XD