Loading ch05/improving_duplicate_file_detector/README-FIX.txt 0 → 100644 +22 −0 Changes for ch05/improving_duplicate_file_detector/README-FIX.txt: 22 added lines, 0 removed lines. Original line number Diff line number Diff line The commands in this section will produce an incorrect result if any image filenames contain space characters. This is because awk treats spaces as input separators. When md5sum produces its two column output, it places two space characters between the checksum and the filename, and awk properly treats them as a separator. But if the filenames also contain space characters, awk incorrectly treats them as separators too. As a result, the second field, $2, is only a prefix of the filename (up to the space), and it is wrongly stored in the names array and printed in the end. A quick fix is to replace the term $2 with substr($0, 35, length), which means "all text from character 35 to the end of the line." The output of md5sum is predictable because the checksums are always 32 characters long. That means the filename begins at character 35. A correct, final command: md5sum *.jpg \ | awk '{counts[$1]++; names[$1]=names[$1] " " substr($0, 35, length) } \ END {for (key in counts) print counts[key] " " key ":" names[key] }' \ | grep -v '^1 ' \ | sort -nr Loading
ch05/improving_duplicate_file_detector/README-FIX.txt 0 → 100644 +22 −0 Changes for ch05/improving_duplicate_file_detector/README-FIX.txt: 22 added lines, 0 removed lines. Original line number Diff line number Diff line The commands in this section will produce an incorrect result if any image filenames contain space characters. This is because awk treats spaces as input separators. When md5sum produces its two column output, it places two space characters between the checksum and the filename, and awk properly treats them as a separator. But if the filenames also contain space characters, awk incorrectly treats them as separators too. As a result, the second field, $2, is only a prefix of the filename (up to the space), and it is wrongly stored in the names array and printed in the end. A quick fix is to replace the term $2 with substr($0, 35, length), which means "all text from character 35 to the end of the line." The output of md5sum is predictable because the checksums are always 32 characters long. That means the filename begins at character 35. A correct, final command: md5sum *.jpg \ | awk '{counts[$1]++; names[$1]=names[$1] " " substr($0, 35, length) } \ END {for (key in counts) print counts[key] " " key ":" names[key] }' \ | grep -v '^1 ' \ | sort -nr