New Perspectives in Information Retrieval: Deep Discovery of Specific Category Websites Driven by Dynamic Keywords

In the digital age, search engines have become the primary means by which users access online information. Identifying and capturing public service websites through search engines pose significant challenges for the Internet, particularly given the frequent updates to website content and adjustments made to search engine algorithms. Traditional methods relying on fixed, single keywords for website discovery fall short in achieving deep excavation and comprehensive coverage of public service websites. This paper innovatively designs a mining method featuring dynamic updating and expansion of the keyword library. First, it employs search engine APIs and web crawling technologies to swiftly gather and consolidate the latest information from various websites. Next, it conducts statistical analysis of keyword frequencies to iteratively generate a set of relevant keywords. Finally, it dynamically adjusts keyword frequencies based on new search outcomes. Experiments demonstrate that our method attains a coverage rate surpassing 95%, outperforming conventional mining approaches. This study provides a robust and highly adaptable framework for the mining of public service websites, offering researchers more precise and timely information retrieval tools. It has been successfully applied in domains such as film, resulting in the establishment of a rich resource database.