Hello there,
I am new to Perceval and exploring its potential for data collection and analysis in quantum computing projects. From what I have gathered so far, Perceval is a great tool for extracting data from a variety of sources, but I am looking for some guidance on how to make the most out of it, particularly when it comes to quantum-related datasets.
My current project involves analyzing quantum algorithm performances over time using both simulation data and real-world experiment results. I am interested in automating the process of fetching relevant papers; datasets; or even forum posts that are related to these algorithms. I have read through the documentation, but I am still unclear on a few things.
How can I configure Perceval to scrape from non-standard sources, such as research repositories that aren’t part of the default integrations? Are there any guides for adding custom sources, especially those related to quantum computing research?
Once I have fetched the data; I would like to apply specific filters related to the quantum algorithms I’m working on. What is the best way to ensure I only collect relevant articles or papers? Is there an efficient way to narrow down results based on certain keywords or research topics?
Also, I have gone through this post; https://perceval.quandela.net/forum/t/massively-slow-response-time-from-this-forum-blue-prism which definitely helped me out a lot.
I plan to integrate the data I collect into a larger pipeline for analysis. Are there any existing integrations with tools like Jupyter notebooks or pandas that might streamline this process?
Thankyou in advance for your help and assistance.