Efficient File Security Scanning: Combining Hash Histories and Bloom Filters to Minimize Redundancy

Efficient File Security Scanning: Combining Hash Histories and Bloom Filters to Minimize Redundancy

Mithilesh Ramaswamy,

Email: rmith87@gmail.com

Abstract

As file repositories expand in size, traditional full-file security scanning becomes computationally expensive and redundant. This paper introduces a novel hybrid framework that leverages hash histories embedded in file metadata and Bloom filters for efficient security scanning. The approach ensures that only modified or newly added files are scanned, reducing overhead while maintaining robust security coverage. By augmenting file metadata with hash histories, the system provides decentralized tracking of file state changes. Bloom filters further optimize the process by efficiently determining whether a file requires scanning. The tradeoff between increased file sizes due to metadata augmentation and the improved scanning performance is thoroughly analyzed. Evaluations demonstrate significant reductions in scan time and resource usage, making the framework highly suitable for large-scale file repositories.

Keywords

File security, Hash histories, Bloom filters, Metadata augmentation, Incremental scanning, Security optimization