如何为相似的输入创建相似的哈希？
我想创建一个包含文件的数据库。而且，为了轻松搜索这些文件，我想使用某种哈希技术。但是，我不仅想找到完全相同的文件，还想检查文件的部分内容是否相同（即文件相似）。换句话说，相似的文件应该具有相似的哈希值。
这意味着这种哈希并不是真正的加密哈希，因为不应该存在“雪崩效应”（雪崩效应意味着每一位数据都会影响其他数据的所有其他位。）
另一件事是哈希不需要是单向的，因为它不是用于安全目的而是用于比较文件。
因此，本质上，我正在寻找一种算法，可以为每个唯一输入创建唯一的哈希：
（几乎）没有冲突
为以下内容创建类似的输出：相似的输入
比原始文件短（否则简单地比较原始文件会更快）。
我正在考虑将前两个字符加在一起，然后将第三个和第四个字符加在一起，等等。但是，这会产生巨大的冲突，因为“1+4”与“2+2”相同，
等等真的不知道如何开始。有人可以启发我吗？ :)
I want to create a database with files. And, to easily search these files, I want to use some kind of hashing technique. However, I don't only want to find files that are EXACTLY the same, but also check if parts of the files are the same(i.e., the files are similar). in other words, similar files should have similar hashes.
This means that this kind of hash is not really a cryptographic hash because there should not be an 'avalanche effect' (avalanche effect means that each bit of data affects ALL other bits of other data.)
Another thing is that the hash does not need to be one-way, since it isn't used for securitypurposes but for the comparing of files.
So in essence, I'm searching for an algorithm that can create an unique hash for each unique input that:
Has (almost) no collision
Creates a similar output for similar inputs
Is shorter than the original file (otherwise it would be faster to simply compare the original files instead).
I was thinking of something like adding the first two characters together, then adding the 3rd and 4rth together, etc. However, this has a HUGE amount of collision since "1+4" is the same as "2+2", etc
I really have no idea how to start. Could somebody enlighten me please? :)
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
这通常称为近似重复检测问题并且不容易解决；我推荐 simhash 算法（代码为 此处）。
This is commonly called the near duplicate detection problem and is not easy to solve; I would recommend the simhash algorithm (code is here).
我目前正在使用 ssdeep 来达到相同的效果，并且我得到了相当好的结果。
我还读到 sdhash 比 ssdeep 更好。
I'm currently using ssdeep to achieve the same effect and I'm getting pretty good results with it.
I've also read that sdhash is better than ssdeep.