All Collections
Content Storage
Web Scraping
Create a web scraping folder and synchronize your data easily
Create a web scraping folder and synchronize your data easily
Promptitude Team avatar
Written by Promptitude Team
Updated over a week ago

In this article, we'll guide you on how to create a web scraping folder and synchronize your data easily with Promptitude. This feature allows you to strengthen your content storage with just a few clicks, extracting information from websites and syncing it seamlessly.

Creating a Web Scraping Folder

To create a web scraping folder, follow these simple steps:

  1. Click on New Folder.

  2. Select Scrape Website.

  3. Add the URL of the home page from which you want to extract information.

Setting Up Your Web Scraping Folder

Once you've created your web scraping folder, you can customize it to suit your needs. Here's how:

βœ… Scraping Interval: This is where you determine the time interval for automatic synchronization. You can set the data synchronization time period according to your preference.

  • Just once

  • Weekly

  • Monthly

  • Quarterly

βœ… Maximum Depth: This setting allows you to set the depth of search and extraction. You can extract information from your main page, subpages, and even legacy pages, all in a branched structure.

  • 1 level

  • 2 levels

  • 3 levels

  • 4 levels

βœ… Maximum Pages to Scrape: Here, you can choose the maximum number of pages you want as content. This helps you control the amount of information you extract.

  • 10 pages

  • 25 pages

  • 50 pages

  • 100 pages

  • 250 pages

  • 500 pages


To make the most of your web scraping folder, consider these recommendations:

  • Re-scrape When Necessary: If you need to update your information, you can re-scrape a content or your entire folder manually.

  • Monitor Your Word Count: Keep an eye on the amount of words consumed by extracting information from websites. This will help you manage your storage effectively.

  • Categorize Your Pages: Use tags to categorize the pages in your web scraping folder. This will help you keep your content organized and easy to find.


Creating a web scraping folder and synchronizing your data with Promptitude is a simple and efficient way to manage your content. By setting up your scraping interval, maximum depth, and maximum pages to scrape, you can customize your folder to suit your needs.

Remember to categorize your pages, monitor your word count, and re-scrape when necessary to keep your content fresh and organized. Happy scraping!

Did this answer your question?