Why Powershell is so slow

performancepowershell

I tried to make a simple thing with PowerShell, find files which take most space on the drive. I used ls + sort and… it took forever for me.

Sometimes I use far manager and compared with PowerShell it looks much faster and stable.

Ok, it is based on the .NET, but .NET is not so slow. I expect to see something lightweight and fast! It is console!

Another thing, I would like to have something like IEnumerable in PowerShell to see results immediately. Is it possible to achieve? It might help a bit while expecting results cause sometimes I think it just hang out.

EDIT

I'm doing something like that

ls -Recurse -ErrorAction SilentlyContinue | sort -Property Size | select -First 10

And I guess it might take DAYS.

EDIT

Just to compare.

C# code took for me about 2 min. For sure it is not ideal and didn't process all files, but it processed at least >95%.

void Main()
{
    GetFilesSize(@"C:\").OrderByDescending(x => x).Take(10).ToList();
}

public IEnumerable<long> GetFilesSize(string directory)
{
    var accessDenied = false;
    var dirList = new string[0]; 
    try
    {
        dirList = Directory.GetDirectories(directory);
    }
    catch{
        accessDenied = true;
    }
    
    if(accessDenied) yield break;
    
    foreach (var dir in dirList)
    {
        foreach (var size in GetFilesSize(dir))
        {
            yield return size;
        }
    }
    
    foreach (var fileName in Directory.GetFiles(directory))
    {
        if(fileName.Length>=260) continue;
        yield return new FileInfo(fileName).Length;
    }
}

Best Answer

PowerShell is a program written in .Net, but it leverages interfaces to many different interpreters and run-times when it's actually running. It's a Shell, so just like BASH, even though it is written in C, that says nothing about the binaries and scripts executed within it. Executables might be .Net code, VDM/CMD commands, *nix shell commands, VB/C/WSScript, WMI invocations, unmanaged API interfaces, jar files, or anything else. These choices are what affect the performance of code running within the shell, not the language the shell is written in.

Now, it sounds like you are having difficulties with the implementation of a specific command. So the better question is, why is ls slow to sort when invoked from within PowerShell. When we dig deeper, we find that ls is an alias for 'Get-ChildItem' which returns an object array containing System.IO.DirectoryInfo objects. 

PS C:\Windows\system32> $x=Get-ChildItem ./
PS C:\Windows\system32> $x.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array 

PS C:\Windows\system32> $x[1].GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     DirectoryInfo                            System.IO.FileSystemInfo   

PS C:\Windows\system32>

You can retrieve the ls result, and then pipe that into a Sort-Object call and it will behave largely the way an IEnumerable does.

Note that IEnumerable doesn't do anything for performance. You may be confusing it with IQueryable, which defines but does not perform a query until the very last second, presumably after it has been decorated with filtering and sorting operations, the way .Net does via LinQ to Objects. In this case, since Get-ChildItem does not offer an optimized query engine or indexed datasource, you cannot really compare modern database operations with directory listings.

So, ultimately, try something like: ls ./ -recurse | Sort-Object Name -descending For me, targeting System32, this takes about 20 seconds to process and sort 54430 files.

Finally, note, that you take a big performance hit when you try to enumerate a directory that you don't personally have access to, so make sure you are not recursing into places you are not allowed to go, or you will suffer a 2+ second wait for each.

Hope that helps.

Related Question