WWW is a repository of hyperlinked data sources that contains heterogeneous data in the form of text, audio, video, and metadata. Search engines play an important role in searching information from these data sources. They offer the users an interface to retrieve the data from these resources. The search engines gather, analyze, organize, and handle the data available in these data sources and return thousands of results in the form of web pages. Often the returned web pages include a mixture of relevant and irrelevant information. To return more significant and relevant response pages, a mechanism to rank the returned pages is desirable. Page ranking algorithms play an important role in ranking web pages so that the user could get the relevant result according to the user query. The proposed technique is used to rank the hidden web pages based on the meta information of the pages downloaded by crawler and the calculated value for the chosen parameters.
TopIntroduction
A lot of information or data is hidden behind the HTML forms on the web. This data is extracted by Hidden Web crawler by automatically filling HTML forms (Bhatia & Gupta, 2012; Gupta & Bhatia, 2014b). Although, the crawlers are made intelligent enough to process the search forms in a specific domain, still the results from the crawlers contain a lot of redundant information and irrelevant results. For Example: when a query such as “job for MBA ” is raised on search engine for Surface Web, many results are returned for jobs for MBA in an unrealistic manner to attract the user attention (Lee et al., 2009). An example of the returned web page is shown in Figure 1.
Figure 1. Result page generated for user’s query by a search engine for the Surface Web
In order to get the more relevant and appropriate results, user has to navigate through the links provided on the result page and when the user proceeds towards the link, these links would turn out to a different link with the same title. query to search “mba” by the user. In case the query is raised at the interface offered by a web database of employment domain as in Figure 2, a list of corresponding result is returned in a web page for the raised query
Figure 2. Interface of hidden web database
However, the Hidden Web pages are dynamically generated based on the query raised in the search form by the crawler, ranking the results become difficult (Gupta & Bhatia, 2017; Rani & Gupta, 2015). The techniques or algorithms for ranking surface web pages are not directly applicable for ranking of Hidden Web pages due to the lack of hyperlink structure in them (Baeza-Yates & Davis, 2004; E, 2012).
TopLiterature Survey
The web page ranking algorithms are applied to rank the search results depending upon their relevance to the search query. The algorithms rank the search results in an order so as to list the most relevant documents at the top of the list amongst the results returned in response to the query string searched in the page returned (E, 2012; Page & Brin, 1998). A web page’s ranking for a specific query depends on factors like- its relevance to the word and concepts in the query, its overall link popularity etc. Page ranking algorithm are applied to calculate rank value of each page. Page rank is a numeric value representing how-important a page is on the web. Web page are sorted according to rank value and are returned to the user. To rank a web page different criteria are used by ranking algorithms. For example some algorithms consider the link structure of the web page while others look for the page content to rank the web page.
The popular PageRank algorithm (Page & Brin, 1998) was developed by Larry Page and Sergey Brin in 1996 uses the link structure of the web to determine the importance of the web page. Here a page obtains a higher rank if sum of its back-links is high. PageRank(PR) is the probability of a page being visited by the user. For each web page, Page Rank value for over 25 billion web pages on the WWW is pre-computed. A Simplified version of PageRank is defined in Equation (1)
(1)where A is a web page whose PageRank is to be calculated. TA is the set of pages that point to A and PR(v) is the rank score of page v that point to A. Qv is the number of links from page v and c is a factor used for normalization (so that the total rank of all web pages is constant). Page Rank algorithm can be applied to a web graph with hyperlinks as shown in Figure 3.