I have a 5-line bit of shellcode that does a pretty good job of it.
In the current version (V4)
$md5 = New-Object -TypeName System.Security.Cryptography.MD5CryptoServiceProvider
$ht = @{}
get-childitem c:\ -Recurse -File |
Select -ExpandProperty FullName |
foreach {
$hash = [System.BitConverter]::ToString($md5.ComputeHash([System.IO.File]::ReadAllBytes($_.FullName)))
$ht[$hash] += @($_.FullName)
}
$ht.GetEnumerator() |
Where {$_.Value.count -gt 1}
V5, which will be released with Windows 10 has a Get-FileHash command that would simplify that down to about half as many lines of code.
So, by what measure is that not nearly as powerful as sed, awk, and grep ( assuming that’s what your 5-line bit of shell code is using to find those duplicate files).