Search the most relevant papers on arXiv with one click. Thanks to ChatGPT, it can be completed in a few days.

For friends who search for papers every day, don’t be too happy if there is a useful search tool, and the efficiency will be greatly improved.

But the actual result is that either the search tool is not powerful, or the keywords you enter do not work. Anyway, you cannot tell the paper you want to find in your mind and the search results. It has nothing to do with it, it's just a huge difference.

The website we are going to introduce below can help you solve the problems encountered in paper search. The website is called arXiv Xplorer, which is specially used for semantic search of papers on arXiv. According to the project’s authors, the site’s internal algorithm uses OpenAI’s latest embedding model to generate search queries for users to find the most relevant papers.

##arXiv Xplorer address: https://arxivxplorer.com/

The project author said: He was deeply impressed by OpenAI's new embedding API, so he wanted to see how the embedding was used in practice. So he spent a few days building the project and so far it's working really well. In addition to this, he wrote 80% of the UI in ChatGPT, used pinecone to store the vector database, and used googlecloud functions to embed queries and perform lookups.

If you want to know more about the embedded model, you can go to the website to view it.

Embedding model: https://openai.com/blog/new-and-improved-embedding-model/

With the help of arXiv Xplorer, you can find the paper you need. Even if your description is very vague, or even just enter a description with no information content such as "interesting ML paper", the engine can help you complete it. As shown below, the query results displayed by the website after entering several keywords.

In the process, you’ll discover things you’ve never seen before compared to traditional search tools like Google or arXiv’s own search interesting papers, in comparison, arXiv Xplorer seems to be more effective.

You can also directly search for similar papers by pasting the arxiv url. For example, the input in the figure below is the address of the paper "A Generalist Agent", and the search results are displayed (red box) , A Generalist Agent has a similarity of 100%, and other search results are expanded in order according to their scores.

In addition, you can also click on the small triangle in the red box in the above picture, and then the interface will become as shown below, showing the paper participants and abstract, and there are two more below Function: "More Like This" will display more similar papers; "View" will link to the paper's homepage on arXiv.

​

Seeing this fully functional website, netizens couldn’t hold back their curiosity and asked, "You OpenAI’s embedded technology is used, but this technology is charged, so how much did you pay for it?” The project author said: "It costs $40 to embed all papers in the CS category (about 500,000 articles)."

​

## Some netizens raised a series of questions about the technology, such as: "Does this website embed all arXiv titles?" The project author said: "He embedded the titles and abstracts of all papers, and initially manually did the cosine similarity and Sorting, but pinecone makes it super easy!"

Some netizens suggested: "The search function of this tool is very good! If you can press It would be cool to view and sort by release date." Regarding this point, the project author said that he will continue to optimize and strive to achieve more perfect functions.

The above is the detailed content of Search the most relevant papers on arXiv with one click. Thanks to ChatGPT, it can be completed in a few days.. For more information, please follow other related articles on the PHP Chinese website!