Automated Ruleset Generation for “HTTPS Everywhere”: Challenges, Implementation, and Insights

Automated Ruleset Generation for “HTTPS Everywhere”: Challenges, Implementation, and Insights

Fares Alharbi, Gautam Siddharth Kashyap, Budoor Ahmad Allehyani
Copyright: © 2024 |Pages: 14
DOI: 10.4018/IJISP.347330
Article PDF Download
Open access articles are freely available for download

Abstract

This paper details the implementation of a Web crawler aimed at automating ruleset construction for “HTTPS Everywhere,” with a goal to convert HTTP URLs to secure HTTPS equivalents for enhanced communication security. Developed within a seven-month timeframe, the crawler faced challenges in verifying HTTPS support, varying based on SSL certificate existence and validity. Successful ruleset creation and testing in Firefox and Chrome, adhering to stylistic standards, demonstrated the potential for effective development. The paper explores improving productivity through alternative libraries like Scrapy and Scrapy Cloud. While certain goals, such as in-depth cryptocurrency analysis and web crawler background reading, were unmet due to time constraints, valuable insights were gained. The conclusion underscores the difficulties, successes, and promises of automating ruleset generation through web crawlers for “HTTPS Everywhere,” offering valuable recommendations for advancing web security.
Article Preview
Top

The paper's main principle is HTTPS Everywhere, a collaborative project of the Tor community, and other organizations that contribute to its development. The fundamental goal of this paper's requirements is to ensure the development of powerful rulesets that significantly impact the open-source initiative while adhering to strict guidelines for clarity, security, compatibility, efficiency, and accurate crediting of the organizations responsible for its development.

Dark Web

Davies (2020) proposed a thorough evaluation of the tools and approaches used by law enforcement in the United Kingdom to combat illegal activity on the dark web. Their suggested approach focuses on using covert policing and hacking strategies in the context of dark-web investigations. The literature of the United States and Australia has addressed this topic. However, it has yet to receive significant attention in the United Kingdom. In their paper, the authors assess the effectiveness of the U.K. investigatory authorities in preventing dark-web crimes and suggest legislative changes and improvements to the Codes of Practice.

Vilić(2017) investigates the growing threat of cyberterrorism and its connection to the dark net or deep web. Her suggested approach highlights the anonymity and accessibility that the internet provides to spread terrorist and criminal ideology, reaching a wider audience than offline approaches. The paper explores the criminological aspects of cyberterrorism, highlighting its cutting-edge psychological-warfare approaches and the potential for widespread repercussions. It investigates numerous definitions of cyberterrorism while acknowledging its complexity.Additionally, it examines international legislation intended to resist this threat, highlighting the necessity of multifaceted global cooperation across international organizations, multilateral platforms, and regional initiatives.

Hayes et al. (2018) present a groundbreaking investigation into the growth of dark-web marketplaces following the rise in popularity of the Silk Road. Their research provides an automated analytical framework employing available technologies from the World Wide Web to recognize the difficulty of participant unmasking. The researchers used a case-study marketplace and an AppleScript-based Web crawler to extract vendor and listing account information. Their approach offers a systematic and practical approach to scraping and analyzing the dark web. The study confirms AppleScript's effectiveness and reveals how well it performs while getting significant amounts of data from the market. The highlighted importance of the framework lies in its role in academic understanding and law-enforcement inquiries into illegal activities on the dark web. Evaluation metrics demonstrate the significant volume of data that supports the recommended approach.

Kavallieros et al. (2021) present an informative paper exploring the differences between the surface, deep, and dark webs and clarifying their interdependencies. They provide an overview of the dark web's characteristics before delving into the architectures of three well-known dark nets—Tor, I2P, and Freenet—and explaining the underlying technologies. Their suggested approach focuses on the fundamental technology challenge around anonymity and examines the effects of police activities on the surface web and the dark web. Dalins et al. (2018) address the constraints of existing studies on dark webs like Tor. They suggest the Tor-use Motivation Model (TMM), a two-dimensional classification scheme for law-enforcement use. This model distinguishes between the site's content and motivation, providing a more complex classification scheme for unethical and unlawful behaviors. The authors' research involved obtaining authorization to crawl Tor for analysis and compiling a sizable dataset with various content. Through manual classification of a corpus containing more than 4,000 distinct Tor pages, the TMM demonstrates its efficacy in detecting criminal activity within the dark web.

Complete Article List

Search this Journal:
Reset
Volume 18: 1 Issue (2024)
Volume 17: 1 Issue (2023)
Volume 16: 4 Issues (2022): 2 Released, 2 Forthcoming
Volume 15: 4 Issues (2021)
Volume 14: 4 Issues (2020)
Volume 13: 4 Issues (2019)
Volume 12: 4 Issues (2018)
Volume 11: 4 Issues (2017)
Volume 10: 4 Issues (2016)
Volume 9: 4 Issues (2015)
Volume 8: 4 Issues (2014)
Volume 7: 4 Issues (2013)
Volume 6: 4 Issues (2012)
Volume 5: 4 Issues (2011)
Volume 4: 4 Issues (2010)
Volume 3: 4 Issues (2009)
Volume 2: 4 Issues (2008)
Volume 1: 4 Issues (2007)
View Complete Journal Contents Listing