• 欢迎访问开心洋葱网站,在线教程,推荐使用最新版火狐浏览器和Chrome浏览器访问本网站,欢迎加入开心洋葱 QQ群
  • 为方便开心洋葱网用户,开心洋葱官网已经开启复制功能!
  • 欢迎访问开心洋葱网站,手机也能访问哦~欢迎加入开心洋葱多维思维学习平台 QQ群
  • 如果您觉得本站非常有看点,那么赶紧使用Ctrl+D 收藏开心洋葱吧~~~~~~~~~~~~~!
  • 由于近期流量激增,小站的ECS没能经的起亲们的访问,本站依然没有盈利,如果各位看如果觉着文字不错,还请看官给小站打个赏~~~~~~~~~~~~~!

python代码用于查找指定具有相同内容的文件

python 水墨上仙 2774次浏览

python代码用于查找指定具有相同内容的文件,可以同时指定多个目录
调用方式:python doublesdetector.py c:\;d:\;e:\ > doubles.txt

# Hello, this script is written in Python - http://www.75271.com
# doublesdetector.py 1.0p
import os, os.path, string, sys, sha
message = """
doublesdetector.py 1.0p
This script will search for files that are identical
(whatever their name/date/time).
  Syntax : python %s <directories>
      where <directories> is a directory or a list of directories
      separated by a semicolon (;)
Examples : python %s c:\windows
           python %s c:\;d:\;e:\ > doubles.txt
           python %s c:\program files > doubles.txt
This script is public domain. Feel free to reuse and tweak it.
The author of this script Sebastien SAUVAGE <sebsauvage at sebsauvage dot net>
http://sebsauvage.net/python/
""" % ((sys.argv[0], )*4)
def fileSHA ( filepath ) :
    """ Compute SHA (Secure Hash Algorythm) of a file.
        Input : filepath : full path and name of file (eg. 'c:\windows\emm386.exe')
        Output : string : contains the hexadecimal representation of the SHA of the file.
                          returns '0' if file could not be read (file not found, no read rights...)
    """
    try:
        file = open(filepath,'rb')
        digest = sha.new()
        data = file.read(65536)
        while len(data) != 0:
            digest.update(data)
            data = file.read(65536)
        file.close()
    except:
        return '0'
    else:
        return digest.hexdigest()
def detectDoubles( directories ):
    fileslist = {}
    # Group all files by size (in the fileslist dictionnary)
    for directory in directories.split(';'):
        directory = os.path.abspath(directory)
        sys.stderr.write('Scanning directory '+directory+'...')
        os.path.walk(directory,callback,fileslist)
        sys.stderr.write('\n')
    sys.stderr.write('Comparing files...')
    # Remove keys (filesize) in the dictionnary which have only 1 file
    for (filesize,listoffiles) in fileslist.items():
        if len(listoffiles) == 1:
            del fileslist[filesize]
    # Now compute SHA of files that have the same size,
    # and group files by SHA (in the filessha dictionnary)
    filessha = {}
    while len(fileslist)>0:
        (filesize,listoffiles) = fileslist.popitem()
        for filepath in listoffiles:
            sys.stderr.write('.')
            sha = fileSHA(filepath)
            if filessha.has_key(sha):
                filessha[sha].append(filepath)
            else:
                filessha[sha] = [filepath]
    if filessha.has_key('0'):
        del filessha['0']
    # Remove keys (sha) in the dictionnary which have only 1 file
    for (sha,listoffiles) in filessha.items():
        if len(listoffiles) == 1:
            del filessha[sha]
    sys.stderr.write('\n')
    return filessha
def callback(fileslist,directory,files):
    sys.stderr.write('.')
    for fileName in files:
        filepath = os.path.join(directory,fileName)
        if os.path.isfile(filepath):
            filesize = os.stat(filepath)[6]
            if fileslist.has_key(filesize):
                fileslist[filesize].append(filepath)
            else:
                fileslist[filesize] = [filepath]
if len(sys.argv)>1 :
    doubles = detectDoubles(" ".join(sys.argv[1:]))
    print 'The following files are identical:'
    print '\n'.join(["----\n%s" % '\n'.join(doubles[filesha]) for filesha in doubles.keys()])
    print '----'
else:
    print message


开心洋葱 , 版权所有丨如未注明 , 均为原创丨未经授权请勿修改 , 转载请注明python代码用于查找指定具有相同内容的文件
喜欢 (0)
加载中……