2025-12-05 22:47:25,904 ocrqa_create_bloom_filter.py:282 INFO: Git version: 3bf39e8
2025-12-05 22:47:25,904 ocrqa_create_bloom_filter.py:284 INFO: Using normalization version: 2
2025-12-05 22:47:25,904 ocrqa_create_bloom_filter.py:288 INFO: Frequency threshold mode: adaptive
2025-12-05 22:47:25,904 ocrqa_create_bloom_filter.py:290 INFO: Adaptive parameters: base_freq=2500.0, exponent=2.0, min=2
2025-12-05 22:47:25,904 ocrqa_create_bloom_filter.py:1063 INFO: Namespace(input_files=['lex/fr/realword_ocr_errors.nw.txt', 'lex/fr/ocr_errors.nw.txt', 'lex/fr/old_spelling.rw.txt', 'lex/fr/modern_spelling.rw.txt', 'lex/fr/morph.rw.txt', 'lex/fr/single_char.rw.txt', 'lex/fr/frwiki.unigram.freq.tsv.bz2'], bloom_path='build.d/fp_prob_0.00005_base_2500.0/ocrqa-wp_v2.0.0-fr.bloom', fp_probability=5e-05, log_level='INFO', log_file='build.d/fp_prob_0.00005_base_2500.0/ocrqa-wp_v2.0.0-fr.bloom.log', config=None, min_frequency=2, single_char_min_frequency=500, frequency_threshold_mode='adaptive', adaptive_base_freq=2500.0, adaptive_exponent=2.0, diagnose_bloom=True, git_version='3bf39e8', log_lexicon='build.d/fp_prob_0.00005_base_2500.0/ocrqa-wp_v2.0.0-fr.bloom.lexicon.txt', version=2, language=None)
2025-12-05 22:47:25,904 ocrqa_create_bloom_filter.py:606 INFO: Starting Bloom Filter creation...
2025-12-05 22:47:25,905 ocrqa_create_bloom_filter.py:469 INFO: Processing nonword file: lex/fr/realword_ocr_errors.nw.txt
2025-12-05 22:47:25,905 ocrqa_create_bloom_filter.py:495 INFO: Exclusion list built: 1 words will be filtered from bloom filter
2025-12-05 22:47:25,905 ocrqa_create_bloom_filter.py:469 INFO: Processing nonword file: lex/fr/ocr_errors.nw.txt
2025-12-05 22:47:25,905 ocrqa_create_bloom_filter.py:495 INFO: Exclusion list built: 22 words will be filtered from bloom filter
2025-12-05 22:47:25,905 ocrqa_create_bloom_filter.py:528 INFO: Processing real-word file: lex/fr/old_spelling.rw.txt
2025-12-05 22:47:25,906 ocrqa_create_bloom_filter.py:528 INFO: Processing real-word file: lex/fr/modern_spelling.rw.txt
2025-12-05 22:47:25,906 ocrqa_create_bloom_filter.py:528 INFO: Processing real-word file: lex/fr/morph.rw.txt
2025-12-05 22:47:27,612 ocrqa_create_bloom_filter.py:528 INFO: Processing real-word file: lex/fr/single_char.rw.txt
2025-12-05 22:47:27,612 ocrqa_create_bloom_filter.py:385 INFO: Processing frequency file: lex/fr/frwiki.unigram.freq.tsv.bz2
2025-12-05 22:47:27,627 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 2: 625
2025-12-05 22:47:27,627 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 1: 2500
2025-12-05 22:47:27,627 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 3: 277
2025-12-05 22:47:27,627 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 4: 156
2025-12-05 22:47:27,628 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 5: 100
2025-12-05 22:47:27,628 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 9: 30
2025-12-05 22:47:27,628 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 6: 69
2025-12-05 22:47:27,628 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 7: 51
2025-12-05 22:47:27,628 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 8: 39
2025-12-05 22:47:27,628 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 10: 25
2025-12-05 22:47:27,629 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 11: 20
2025-12-05 22:47:27,629 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 12: 17
2025-12-05 22:47:27,629 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 13: 14
2025-12-05 22:47:27,630 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 14: 12
2025-12-05 22:47:27,631 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 16: 9
2025-12-05 22:47:27,633 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 15: 11
2025-12-05 22:47:27,641 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 18: 7
2025-12-05 22:47:27,649 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 17: 8
2025-12-05 22:47:27,661 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 20: 6
2025-12-05 22:47:27,665 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 23: 4
2025-12-05 22:47:27,683 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 19: 6
2025-12-05 22:47:27,685 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 21: 5
2025-12-05 22:47:27,694 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 25: 4
2025-12-05 22:47:27,704 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 28: 3
2025-12-05 22:47:27,709 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 29: 2
2025-12-05 22:47:27,713 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 22: 5
2025-12-05 22:47:27,722 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 27: 3
2025-12-05 22:47:27,761 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 34: 2
2025-12-05 22:47:27,782 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 24: 4
2025-12-05 22:47:27,802 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 26: 3
2025-12-05 22:47:27,883 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 31: 2
2025-12-05 22:47:27,909 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 30: 2
2025-12-05 22:47:28,007 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 42: 2
2025-12-05 22:47:28,106 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 32: 2
2025-12-05 22:47:28,129 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 44: 2
2025-12-05 22:47:28,225 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 35: 2
2025-12-05 22:47:28,299 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 33: 2
2025-12-05 22:47:28,557 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 36: 2
2025-12-05 22:47:28,613 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 40: 2
2025-12-05 22:47:28,613 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 37: 2
2025-12-05 22:47:28,714 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 53: 2
2025-12-05 22:47:28,951 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 41: 2
2025-12-05 22:47:29,038 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 64: 2
2025-12-05 22:47:29,452 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 47: 2
2025-12-05 22:47:29,680 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 43: 2
2025-12-05 22:47:29,711 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 38: 2
2025-12-05 22:47:30,009 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 146: 2
2025-12-05 22:47:30,098 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 39: 2
2025-12-05 22:47:30,199 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 51: 2
2025-12-05 22:47:30,251 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 45: 2
2025-12-05 22:47:30,660 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 57: 2
2025-12-05 22:47:30,700 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 46: 2
2025-12-05 22:47:30,812 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 52: 2
2025-12-05 22:47:31,292 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 69: 2
2025-12-05 22:47:31,860 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 54: 2
2025-12-05 22:47:31,999 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 126: 2
2025-12-05 22:47:32,128 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 83: 2
2025-12-05 22:47:32,179 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 58: 2
2025-12-05 22:47:32,198 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 67: 2
2025-12-05 22:47:32,738 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 121: 2
2025-12-05 22:47:32,872 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 100: 2
2025-12-05 22:47:32,935 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 49: 2
2025-12-05 22:47:32,983 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 68: 2
2025-12-05 22:47:33,039 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 50: 2
2025-12-05 22:47:33,061 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 85: 2
2025-12-05 22:47:33,234 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 55: 2
2025-12-05 22:47:33,282 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 108: 2
2025-12-05 22:47:33,297 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 48: 2
2025-12-05 22:47:33,687 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 63: 2
2025-12-05 22:47:34,083 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 70: 2
2025-12-05 22:47:34,129 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 104: 2
2025-12-05 22:47:34,210 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 62: 2
2025-12-05 22:47:34,298 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 93: 2
2025-12-05 22:47:34,416 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 86: 2
2025-12-05 22:47:34,419 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 65: 2
2025-12-05 22:47:34,434 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 188: 2
2025-12-05 22:47:34,442 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 99: 2
2025-12-05 22:47:34,678 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 97: 2
2025-12-05 22:47:34,780 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 94: 2
2025-12-05 22:47:34,786 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 89: 2
2025-12-05 22:47:34,825 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 87: 2
2025-12-05 22:47:34,859 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 66: 2
2025-12-05 22:47:34,865 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 59: 2
2025-12-05 22:47:34,907 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 78: 2
2025-12-05 22:47:34,943 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 74: 2
2025-12-05 22:47:35,046 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 109: 2
2025-12-05 22:47:35,090 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 124: 2
2025-12-05 22:47:35,169 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 88: 2
2025-12-05 22:47:35,235 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 76: 2
2025-12-05 22:47:35,289 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 75: 2
2025-12-05 22:47:35,345 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 101: 2
2025-12-05 22:47:35,652 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 60: 2
2025-12-05 22:47:35,779 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 144: 2
2025-12-05 22:47:35,975 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 140: 2
2025-12-05 22:47:36,072 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 111: 2
2025-12-05 22:47:36,075 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 117: 2
2025-12-05 22:47:36,089 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 73: 2
2025-12-05 22:47:36,136 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 71: 2
2025-12-05 22:47:36,540 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 102: 2
2025-12-05 22:47:36,682 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 105: 2
2025-12-05 22:47:36,718 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 136: 2
2025-12-05 22:47:36,807 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 80: 2
2025-12-05 22:47:36,816 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 178: 2
2025-12-05 22:47:36,822 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 56: 2
2025-12-05 22:47:36,823 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 84: 2
2025-12-05 22:47:36,824 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 141: 2
2025-12-05 22:47:36,829 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 120: 2
2025-12-05 22:47:36,869 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 106: 2
2025-12-05 22:47:36,884 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 123: 2
2025-12-05 22:47:36,885 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 82: 2
2025-12-05 22:47:36,899 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 79: 2
2025-12-05 22:47:36,901 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 98: 2
2025-12-05 22:47:36,901 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 61: 2
2025-12-05 22:47:36,905 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 90: 2
2025-12-05 22:47:36,911 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 122: 2
2025-12-05 22:47:36,923 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 181: 2
2025-12-05 22:47:36,930 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 138: 2
2025-12-05 22:47:36,961 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 81: 2
2025-12-05 22:47:36,963 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 96: 2
2025-12-05 22:47:36,973 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 129: 2
2025-12-05 22:47:36,974 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 116: 2
2025-12-05 22:47:36,980 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 277: 2
2025-12-05 22:47:36,999 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 92: 2
2025-12-05 22:47:36,999 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 114: 2
2025-12-05 22:47:37,012 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 195: 2
2025-12-05 22:47:37,013 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 156: 2
2025-12-05 22:47:37,028 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 112: 2
2025-12-05 22:47:37,036 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 72: 2
2025-12-05 22:47:37,049 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 118: 2
2025-12-05 22:47:37,078 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 154: 2
2025-12-05 22:47:37,091 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 147: 2
2025-12-05 22:47:37,152 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 272: 2
2025-12-05 22:47:37,163 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 77: 2
2025-12-05 22:47:37,220 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 128: 2
2025-12-05 22:47:37,228 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 91: 2
2025-12-05 22:47:37,228 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 130: 2
2025-12-05 22:47:37,233 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 169: 2
2025-12-05 22:47:37,261 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 125: 2
2025-12-05 22:47:37,314 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 115: 2
2025-12-05 22:47:37,322 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 103: 2
2025-12-05 22:47:37,327 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 170: 2
2025-12-05 22:47:37,344 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 189: 2
2025-12-05 22:47:37,356 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 139: 2
2025-12-05 22:47:37,397 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 119: 2
2025-12-05 22:47:37,468 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 113: 2
2025-12-05 22:47:37,496 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 110: 2
2025-12-05 22:47:37,522 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 95: 2
2025-12-05 22:47:37,588 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 153: 2
2025-12-05 22:47:37,595 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 133: 2
2025-12-05 22:47:37,640 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 131: 2
2025-12-05 22:47:37,670 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 163: 2
2025-12-05 22:47:37,721 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 289: 2
2025-12-05 22:47:37,803 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 158: 2
2025-12-05 22:47:37,814 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 185: 2
2025-12-05 22:47:37,864 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 150: 2
2025-12-05 22:47:37,979 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 107: 2
2025-12-05 22:47:37,981 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 167: 2
2025-12-05 22:47:38,001 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 157: 2
2025-12-05 22:47:38,004 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 213: 2
2025-12-05 22:47:38,005 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 132: 2
2025-12-05 22:47:38,057 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 135: 2
2025-12-05 22:47:38,076 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 127: 2
2025-12-05 22:47:38,080 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 173: 2
2025-12-05 22:47:38,123 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 174: 2
2025-12-05 22:47:38,192 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 182: 2
2025-12-05 22:47:38,269 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 236: 2
2025-12-05 22:47:38,302 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 152: 2
2025-12-05 22:47:38,303 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 225: 2
2025-12-05 22:47:38,439 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 143: 2
2025-12-05 22:47:38,500 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 134: 2
2025-12-05 22:47:38,537 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 355: 2
2025-12-05 22:47:38,601 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 180: 2
2025-12-05 22:47:38,620 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 262: 2
2025-12-05 22:47:38,659 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 177: 2
2025-12-05 22:47:38,747 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 247: 2
2025-12-05 22:47:38,762 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 165: 2
2025-12-05 22:47:38,853 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 155: 2
2025-12-05 22:47:38,859 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 224: 2
2025-12-05 22:47:38,863 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 149: 2
2025-12-05 22:47:38,903 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 148: 2
2025-12-05 22:47:38,974 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 260: 2
2025-12-05 22:47:38,985 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 164: 2
2025-12-05 22:47:39,026 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 172: 2
2025-12-05 22:47:39,124 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 171: 2
2025-12-05 22:47:39,127 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 145: 2
2025-12-05 22:47:39,182 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 184: 2
2025-12-05 22:47:39,214 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 142: 2
2025-12-05 22:47:39,299 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 199: 2
2025-12-05 22:47:39,675 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 161: 2
2025-12-05 22:47:39,803 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 257: 2
2025-12-05 22:47:39,823 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 222: 2
2025-12-05 22:47:39,937 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 197: 2
2025-12-05 22:47:40,011 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 230: 2
2025-12-05 22:47:40,542 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 246: 2
2025-12-05 22:47:40,592 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 162: 2
2025-12-05 22:47:40,762 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 379: 2
2025-12-05 22:47:40,784 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 201: 2
2025-12-05 22:47:40,817 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 159: 2
2025-12-05 22:47:40,980 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 168: 2
2025-12-05 22:47:40,984 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 183: 2
2025-12-05 22:47:40,995 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 251: 2
2025-12-05 22:47:41,002 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 219: 2
2025-12-05 22:47:41,100 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 137: 2
2025-12-05 22:47:41,195 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 243: 2
2025-12-05 22:47:41,396 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 348: 2
2025-12-05 22:47:41,697 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 245: 2
2025-12-05 22:47:41,889 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 231: 2
2025-12-05 22:47:42,116 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 166: 2
2025-12-05 22:47:42,385 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 151: 2
2025-12-05 22:47:43,005 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 176: 2
2025-12-05 22:47:43,033 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 342: 2
2025-12-05 22:47:43,322 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 302: 2
2025-12-05 22:47:43,358 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 191: 2
2025-12-05 22:47:43,813 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 375: 2
2025-12-05 22:47:44,016 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 352: 2
2025-12-05 22:47:44,377 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 186: 2
2025-12-05 22:47:44,454 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 200: 2
2025-12-05 22:47:44,635 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 175: 2
2025-12-05 22:47:44,985 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 190: 2
2025-12-05 22:47:45,489 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 187: 2
2025-12-05 22:47:45,529 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 160: 2
2025-12-05 22:47:45,530 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 248: 2
2025-12-05 22:47:45,973 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 203: 2
2025-12-05 22:47:45,990 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 274: 2
2025-12-05 22:47:46,788 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 415: 2
2025-12-05 22:47:46,802 ocrqa_create_bloom_filter.py:360 INFO: Frequency threshold for word length 212: 2
2025-12-05 22:47:46,920 ocrqa_create_bloom_filter.py:623 INFO: Low-frequency exclusions before cleanup: 3045207 tokens
2025-12-05 22:47:46,983 ocrqa_create_bloom_filter.py:631 INFO: Low-frequency exclusions after cleanup: 2802564 tokens
2025-12-05 22:47:46,983 ocrqa_create_bloom_filter.py:635 INFO: Lexical processing complete.
2025-12-05 22:47:46,983 ocrqa_create_bloom_filter.py:638 INFO: Processing Statistics:
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   freq_words_filtered: 3,772,357
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   freq_words_read: 4,047,539
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   low_freq_excluded: 2,802,564
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   nonwords_count: 22
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   nonwords_read: 22
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   realwords_accepted: 723,460
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   realwords_nonwords_filtered: 1
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   realwords_read: 723,461
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   single_char_words_filtered: 7,044
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   subwords_accepted: 282,162
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   subwords_filtered: 152
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:640 INFO:   words_accepted: 268,138
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:644 INFO: Creating bloom filter with 866,860 unique words/tokens
2025-12-05 22:47:46,984 ocrqa_create_bloom_filter.py:647 INFO: Target false positive probability: 5e-05
2025-12-05 22:47:47,170 ocrqa_create_bloom_filter.py:652 INFO: Bloom filter created successfully: build.d/fp_prob_0.00005_base_2500.0/ocrqa-wp_v2.0.0-fr.bloom
2025-12-05 22:47:47,170 ocrqa_create_bloom_filter.py:723 INFO: Writing metadata to build.d/fp_prob_0.00005_base_2500.0/ocrqa-wp_v2.0.0-fr.bloom.json...
2025-12-05 22:47:47,171 ocrqa_create_bloom_filter.py:768 INFO: Metadata written to build.d/fp_prob_0.00005_base_2500.0/ocrqa-wp_v2.0.0-fr.bloom.json
2025-12-05 22:47:47,171 ocrqa_create_bloom_filter.py:685 INFO: Writing lexicon to build.d/fp_prob_0.00005_base_2500.0/ocrqa-wp_v2.0.0-fr.bloom.lexicon.txt...
2025-12-05 22:47:47,563 ocrqa_create_bloom_filter.py:689 INFO: Lexicon written to build.d/fp_prob_0.00005_base_2500.0/ocrqa-wp_v2.0.0-fr.bloom.lexicon.txt (866860 words)
2025-12-05 22:47:47,697 ocrqa_create_bloom_filter.py:816 INFO: Diagnosis Results:
2025-12-05 22:47:47,697 ocrqa_create_bloom_filter.py:817 INFO:   - Excluded words in bloom filter: 0
2025-12-05 22:47:47,697 ocrqa_create_bloom_filter.py:818 INFO:   - Known words not in bloom filter: 0
2025-12-05 22:47:48,059 ocrqa_create_bloom_filter.py:825 INFO:   - Low-frequency words in bloom filter: 134
2025-12-05 22:47:48,059 ocrqa_create_bloom_filter.py:831 INFO:   - Proportion of excluded words in bloom filter: 0.00000000
2025-12-05 22:47:48,059 ocrqa_create_bloom_filter.py:837 INFO:   - Proportion of known words not in bloom filter: 0.00000000
2025-12-05 22:47:48,059 ocrqa_create_bloom_filter.py:845 INFO:   - Proportion of low-frequency words in bloom filter: 0.00004781