Powershell Copy-Item 但仅复制更改的文件

Question

我正在尝试递归一个目录并将其从 A 复制到 B。这可以通过以下方式完成：

Copy-Item C:\MyTest C:\MyTest2 –recurse

我希望能够只复制新文件（存在于 src 中但不存在于 dest 中的文件）并且也只复制文件这可能是基于 CRC 检查而不是日期时间戳而改变的。

$file = "c:\scripts"
param
(
$file
)

$algo = [System.Security.Cryptography.HashAlgorithm]::Create("MD5")
$stream = New-Object System.IO.FileStream($file, [System.IO.FileMode]::Open)

$md5StringBuilder = New-Object System.Text.StringBuilder
$algo.ComputeHash($stream) | `
% { [void] $md5StringBuilder.Append($_.ToString("x2")) }
$md5StringBuilder.ToString()

$stream.Dispose() 

这段代码为我提供了对特定文件的 CRC 检查...我只是不确定如何将两个脚本放在一起才能真正满足我的需要。 我也不知道上面的 CRC 检查是否实际上是执行此操作的正确方法。

有人有任何见解吗？

Answer 1

我找到了一个解决方案......但不确定从性能角度来看它是最好的：

$Source = "c:\scripts"
$Destination = "c:\test"
###################################################
###################################################
Param($Source,$Destination)
function Get-FileMD5 {
    Param([string]$file)
    $mode = [System.IO.FileMode]("open")
    $access = [System.IO.FileAccess]("Read")
    $md5 = New-Object System.Security.Cryptography.MD5CryptoServiceProvider
    $fs = New-Object System.IO.FileStream($file,$mode,$access)
    $Hash = $md5.ComputeHash($fs)
    $fs.Close()
    [string]$Hash = $Hash
    Return $Hash
}
function Copy-LatestFile{
    Param($File1,$File2,[switch]$whatif)
    $File1Date = get-Item $File1 | foreach-Object{$_.LastWriteTimeUTC}
    $File2Date = get-Item $File2 | foreach-Object{$_.LastWriteTimeUTC}
    if($File1Date -gt $File2Date)
    {
        Write-Host "$File1 is Newer... Copying..."
        if($whatif){Copy-Item -path $File1 -dest $File2 -force -whatif}
        else{Copy-Item -path $File1 -dest $File2 -force}
    }
    else
    {
        #Don't want to copy this in my case..but good to know
        #Write-Host "$File2 is Newer... Copying..."
        #if($whatif){Copy-Item -path $File2 -dest $File1 -force -whatif}
        #else{Copy-Item -path $File2 -dest $File1 -force}
    }
    Write-Host
}

# Getting Files/Folders from Source and Destination
$SrcEntries = Get-ChildItem $Source -Recurse
$DesEntries = Get-ChildItem $Destination -Recurse

# Parsing the folders and Files from Collections
$Srcfolders = $SrcEntries | Where-Object{$_.PSIsContainer}
$SrcFiles = $SrcEntries | Where-Object{!$_.PSIsContainer}
$Desfolders = $DesEntries | Where-Object{$_.PSIsContainer}
$DesFiles = $DesEntries | Where-Object{!$_.PSIsContainer}

# Checking for Folders that are in Source, but not in Destination
foreach($folder in $Srcfolders)
{
    $SrcFolderPath = $source -replace "\\","\\" -replace "\:","\:"
    $DesFolder = $folder.Fullname -replace $SrcFolderPath,$Destination
    if(!(test-path $DesFolder))
    {
        Write-Host "Folder $DesFolder Missing. Creating it!"
        new-Item $DesFolder -type Directory | out-Null
    }
}

# Checking for Folders that are in Destinatino, but not in Source
foreach($folder in $Desfolders)
{
    $DesFilePath = $Destination -replace "\\","\\" -replace "\:","\:"
    $SrcFolder = $folder.Fullname -replace $DesFilePath,$Source
    if(!(test-path $SrcFolder))
    {
        Write-Host "Folder $SrcFolder Missing. Creating it!"
        new-Item $SrcFolder -type Directory | out-Null
    }
}

# Checking for Files that are in the Source, but not in Destination
foreach($entry in $SrcFiles)
{
    $SrcFullname = $entry.fullname
    $SrcName = $entry.Name
    $SrcFilePath = $Source -replace "\\","\\" -replace "\:","\:"
    $DesFile = $SrcFullname -replace $SrcFilePath,$Destination
    if(test-Path $Desfile)
    {
        $SrcMD5 = Get-FileMD5 $SrcFullname
        $DesMD5 = Get-FileMD5 $DesFile
        If(Compare-Object $srcMD5 $desMD5)
        {
            Write-Host "The Files MD5's are Different... Checking Write
            Dates"
            Write-Host $SrcMD5
            Write-Host $DesMD5
            Copy-LatestFile $SrcFullname $DesFile
        }
    }
    else
    {
        Write-Host "$Desfile Missing... Copying from $SrcFullname"
        copy-Item -path $SrcFullName -dest $DesFile -force
    }
}

# Checking for Files that are in the Destinatino, but not in Source
foreach($entry in $DesFiles)
{
    $DesFullname = $entry.fullname
    $DesName = $entry.Name
    $DesFilePath = $Destination -replace "\\","\\" -replace "\:","\:"
    $SrcFile = $DesFullname -replace $DesFilePath,$Source
    if(!(test-Path $SrcFile))
    {
        Write-Host "$SrcFile Missing... Copying from $DesFullname"
        copy-Item -path $DesFullname -dest $SrcFile -force
    }
}

Answer 2

它有点长，但它的工作效果令人钦佩 - 可以扩展以查找文件的存档位 ---a-- 属性。 无论如何，对于某人来说可能是一个合理的起点。

Function GetFileSHA ($file) {
    return [Array](Get-FileHash $file -Algorithm SHA256);
}
$SourceDir = "C:\temp\1";
$TargetDir = "C:\temp\2";
$SourceFiles = Get-ChildItem -Recurse $SourceDir;
$TargetFiles = Get-ChildItem -Recurse $TargetDir;

#Source Table
$dt = New-Object System.Data.DataTable;
$dt.TableName = "SrcFiles";
$dtcol1 = New-Object system.Data.DataColumn fileId,([System.Int32]); $dt.columns.add($dtcol1);
$dtcol1.AllowDBNull = $false;
$dtcol1.AutoIncrement = $true;
$dtcol1.AutoIncrementSeed = 0;
$dtcol1.Unique = $true;
$dt.PrimaryKey = $dtcol1;
$dtcol2 = New-Object system.Data.DataColumn fileName,([string]); $dt.columns.add($dtcol2);
$dtcol3 = New-Object system.Data.DataColumn filePath,([string]); $dt.columns.add($dtcol3);
$dtcol3.Unique = $true;
$dtcol4 = New-Object system.Data.DataColumn fileHash,([string]); $dt.columns.add($dtcol4);
$dtcol5 = New-Object system.Data.DataColumn fileDate,([System.DateTime]); $dt.columns.add($dtcol5);

#Target Table
$dt2 = New-Object System.Data.DataTable;
$dt2.TableName = "TrgFiles";
$dt2col1 = New-Object system.Data.DataColumn fileId,([System.Int32]); $dt2.columns.add($dt2col1);
$dt2col1.AllowDBNull = $false;
$dt2col1.AutoIncrement = $true;
$dt2col1.AutoIncrementSeed = 0;
$dt2col1.Unique = $true;
$dt2.PrimaryKey = $dt2col1;
$dt2col2 = New-Object system.Data.DataColumn fileName,([string]); $dt2.columns.add($dt2col2);
$dt2col3 = New-Object system.Data.DataColumn filePath,([string]); $dt2.columns.add($dt2col3);
$dt2col3.Unique = $true;
$dt2col4 = New-Object system.Data.DataColumn fileHash,([string]); $dt2.columns.add($dt2col4);
$dt2col5 = New-Object system.Data.DataColumn fileDate,([System.DateTime]); $dt2.columns.add($dt2col5);

#Store file hashes and other attributes into DataTable for comparison
ForEach ($src_file in $SourceFiles){
    $this_file = GetFileSHA $src_file.FullName;
    $row = $dt.NewRow();
    $row.fileName=($src_file).PSChildName;
    $row.filePath=($src_file).FullName;
    $row.fileHash=($this_file).Hash;
    $row.fileDate=($src_file).LastWriteTimeUtc;
    $dt.Rows.Add($row); 
}
ForEach ($trg_file in $TargetFiles){
    $this_file = GetFileSHA $trg_file.FullName;
    $row = $dt2.NewRow();
    $row.fileName=($trg_file).PSChildName;
    $row.filePath=($trg_file).FullName;
    $row.fileHash=($this_file).Hash;
    $row.fileDate=($trg_file).LastWriteTimeUtc;
    $dt2.Rows.Add($row);    
}

#Compare and copy if newer/changed
ForEach ($file in $dt){
    $search_dt2 = ($dt2 | Select-Object "fileName", "filePath", "fileDate", "fileHash" | Where-Object {$_.fileName -eq $file.fileName})
    if ($file.fileHash -eq $search_dt2.fileHash){
        $result=1;
        Write-Host "File Hashes are a match - checking LastWrite status just to be safe...";
    } else {
        $result=2;
        Write-Host "File Hashes are not a match - checking LastWrite status to see if Target is newer than source...";
    }
    if ($result -eq 1 -and ($file.fileDate -eq $search_dt2.fileDate)){
        $result=1;
        Write-Host "LastWrite status is a match. File will be skipped from copy.";
    } elseif ($result -eq 1 -and $search_dt2.fileDate -gt $file.fileDate) {
        $result=1;
        Write-Host "LastWrite status of the target is newer. File will be skipped from copy.";
    } elseif ($result -ne 1 -and $search_dt2.fileDate -gt $file.fileDate) {
        $result=1;
        Write-Host "LastWrite status of the target is newer than the source. File will be skipped from copy.";
    } else {
        $result=3;
        Write-Host "File in target is older than the source and is scheduled for copy...";
    }
    if (Test-Path $search_dt2.filePath){
    } else {
        $result=4;
        Write-Host "File does not exist in the target folder - file is scheduled for copy...";
    }
    
    #DO the action based on above logic
    if($result -ne 1){
        Copy-Item -Path $file.filePath -Destination $search_dt2.filePath -Force -Verbose
        Write-Host "Code:[$result]";        
    }   else {
        Write-Host "Code:[$result]";
    }
}

Answer 3

尝试下面的方法。 它将在 7 天内检查任何内容，您可以设置或不设置。 但是，它会查找任何不存在的文件或过去 7 天内更新的任何内容。

$sourcePath = "C:\MyTest"
$destinationPath = "C:\MyTest2"

function Get-FileCRC32 {
    param (
        [string]$filePath
    )
    
    $fileContent = Get-Content -Path $filePath -Raw
    $hashAlgorithm = [System.Security.Cryptography.CryptoConfig]::CreateFromName("CRC32")
    $hashProvider = [System.Security.Cryptography.HashAlgorithm]::Create($hashAlgorithm)

    $hashBytes = $hashProvider.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($fileContent))
    $hashValue = [BitConverter]::ToString($hashBytes) -replace '-'

    return $hashValue
}

Get-ChildItem -Path $sourcePath -Recurse -File -Exclude *.log | 
    Where-Object {
        !($_.PSIsContainer) -and 
        (Test-Path -LiteralPath (Join-Path $destinationPath $_.FullName.Substring($sourcePath.length))) -eq $false -or
        (Get-FileCRC32 $_.FullName) -ne (Get-FileCRC32 (Join-Path $destinationPath $_.FullName.Substring($sourcePath.length)))
    } | 
    Copy-Item -Destination $destinationPath -Force -Verbose -Confirm:$false

Answer 4

这两个都是 powershell 的可靠答案，但利用 Robocopy 可能会容易得多（MS 提供了强大的复制应用程序）。

robocopy "C:\SourceDir\" "C:\DestDir\" /MIR

会完成同样的事情。

Answer 5

以下是一些如何使脚本更易于维护的指南。

将原始脚本转换为过滤器。

filter HasChanged { 
    param($file)

    # if $file's MD5 has does not exist
    # then return $_
}

然后简单地过滤所有更新的文件并复制它们。

# Note that "Copy-Item" here does not preserve original directory structure
# Every updated file gets copied right under "C:\MyTest2"
ls C:\MyTest -Recurse | HasChanged | Copy-Item -Path {$_} C:\MyTest2

或者您可以创建另一个生成子目录的函数

ls C:\MyTest -Recurse | HasChanged | % { Copy-Item $_ GenerateSubDirectory(...) }