Web Crawler
The Project
The client, a leading online shopping firm, outsourced the Web Crawling project to Tricom, to crawl sites that have different complexities. With a new change in developing web applications came a whole set of new challenges that their existing crawler failed to render.
The Challenge
The major challenges confronting the client were:
- Crawling sites that have JavaScript/AJAX.
- Defined time for crawling big sites.
The Solution
Tricom’s crawling assisted the client in overcoming the challenges by:
- Developing customized crawlers that were site specific. We had performed an experiment of running our crawling framework over a number of representative AJAX sites. We analyzed the overall performance of our approach, evaluated the effectiveness in retrieving relevant clicks, assessed the quality and correctness of scraping contents and examined the capability of our crawlers on real sites used in practice.
- Smartly balancing the load, thereby reducing the total time for crawling the huge site.
HighlightsSpecialized in electronic management of business documents
Web Crawler: Case Studies
The client, a leading online shopping firm, outsourced the Web Crawling project to Tricom for crawling sites that have different major complexities … more++