1/4/2024 0 Comments Octoparse legitSometimes you need to find alternate ones. hidden behind an 'Display' Ajax button that I wasn't able to deal with (with php / cUrl) 10 tasks are offered for free, and as far I know, won't be public tasks as it's the case with some of Octoparse competitors Smart Mode and Wizard mode make it easy to find the data, often at first sight. because I was unable to access the most important part of the data I needed. as if it wouldn't be any ajax routines on the pages. Several reasons for it : easy to set up lots of tutorials to start easily Ajax is handled as easy as a basic html url. had to be fast, had to be robust ! I gave a try to some scraping tools, and my final choice was made to Octoparse. In two word : a nightmare ! So, I had to find a way to still be able to extract my needed data, without having to pass an engineer degree in information technology. and the dynamic pages that don't load at first sight, that wait for you to click on a button, that just show as you scroll down, that exchange static pictures urls with javascipt dynamically shown pictures. Then came for me (and I must admit, my limited skills) THE hammer : AJAX ! Yes, html + Javascipt + css + dom. In fact, websites regularly change minor things on their pages, and in the best case, you wouldn't get anymore some or all of the awaited data, in the worse case, absolutely inaccurate data. Years after years, it sounded clear that my extracting routines running on my server were more and more difficult to maintain in a good working shape. I have been crawling and parsing websites for a while, with use of php and cUrl. Issues with not loading, white pages, crashing have been minimized.I wish I had discovered this jewel years ago. Added Zapier integration to transfer cloud data to a third-party app such as Google Sheets, Dropbox, etc.Back up local data to the Cloud (for Professional users and above).Allows custom user agents(UA) for browser options.Allows timestamp format for datetime conversion.Allows scrolling the webpage partially within a specific area for websites with several scroll bars.Capture data dynamically as the page scrolls if data is not being extracted properly for infinitive-scrolling websites, such as Twitter.Updated database auto-export tool to help better manage export schedules with filters and sorts.Updated task edit settings, workflow and data preview interface.Updated "Data Preview" session with newly added vertical view for data fields and different page layers to view data fields.Improved core browser performance for local runs and task edit mode with much better stability and compatibility.Fixed a bug where the app becomes unresponsive or returns a blank screen after a long-time usage.Fixed a bug which leads to frozen export window or unresponsive export button.Supports file formats including image, audio, video, document, etcĪdded the ability to solve three major types of CAPTCHAs automaticallyĪdded a built-in proxy service that supports access from multiple geographical locations.Adds the ability to download files from web pages while scraping data.Users can add an action to go back to a previous page even when the webpage does not offer an button. Users can now enter multiple text strings into multiple text boxes with one workflow action. Users can now complete transactions inside the app and use prepaid credits on templates, Octoparse proxies, and CAPTCHA solving service. Brought back an old feature to view errors and retry failed URLs after a local run is finished. ![]() Upgraded “Cloud Live Log” to “Cloud Run Window”.This update also contains minor bug fixes and performance improvements.We'll be using Intercom to send out more relevant resources. Automated announcements and messages based on user activity. ![]() ![]() Once a ticket is created, users will be able to track the progress of that ticket through notifications in the widget, as well as, through email notifications as the ticket is updated. Tickets can now be created directly inside the chat widget through a guided process.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |