Performance Issue

[Azi Hassan via Digitalmars-d-learn]

On Wednesday, 6 September 2017 at 15:11:57 UTC, Vino.B wrote:
> On Wednesday, 6 September 2017 at 14:38:39 UTC, Vino.B wrote:
> Hi Azi,
>
>   The required out is like below
>
> [31460]  - Array 1 for folder 1(all files in Folder 1) of the 
> FS C:\\Temp\\TEST1\\BACKUP
> [1344448]  - Array 2 for folder 2(all files in Folder 2) of the 
> FS C:\\Temp\\TEST1\\BACKUP
> [2277663, 2277663]  - Array 3 for folder 1(all files in Folder 
> 1) of the FS C:\\Temp\\TEST2\\EXPOR
> [31460] - Array 4 for folder 2(all files in Folder 2) the FS 
> C:\\Temp\\TEST2\\EXPORT

I tried to create a similar file structure on my Linux machine. 
Here's the result of ls -R TEST1:

TEST1:
BACKUP

TEST1/BACKUP:
FOLDER1
FOLDER2

TEST1/BACKUP/FOLDER1:
file1
file2
file3

TEST1/BACKUP/FOLDER2:
b1
b2

And here's the output of ls -R TEST2 :

TEST2:
EXPORT

TEST2/EXPORT:
FOLDER1
FOLDER2

TEST2/EXPORT/FOLDER1:
file2_1
file2_2
file2_3

TEST2/EXPORT/FOLDER2:
export1
export2
export3
export4

This codes output the sizes in the format you described :

import std.algorithm: filter, map, fold, each;
import std.parallelism: parallel;
import std.file: SpanMode, dirEntries, DirEntry;
import std.stdio: writeln;
import std.typecons: tuple;
import std.path: globMatch;
import std.array;

void main () {
	auto Filesys = ["TEST1/BACKUP", "TEST2/EXPORT"];
	ulong[][] sizes;
	foreach(FFs; Filesys)
	{
		auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a => 
a.isDir).map!(a => a.name);
		foreach (d; dFiles) {
			sizes ~= dirEntries(d, SpanMode.depth).map!(a => a.size).array;
		}
	}
	sizes.each!writeln;
}

It outputs the sizes :

[6, 6, 6]
[8, 8]
[8, 8, 8]
[9, 9, 9, 9]

Note that there's no need to store them in ulong[][] sizes, you 
can display them inside the loop by replacing `sizes ~= 
dirEntries(d, SpanMode.depth).map!(a => a.size).array;` with 
`dirEntries(d, SpanMode.depth).map!(a => a.size).joiner(", 
").writeln;`

To make sure that it calculates the correct sizes, I made it 
display the paths instead by making "sizes" string[][] instead of 
ulong[][] and by replacing map!(a => a.size) with map!(a => 
a.name) in the second foreach loop :

import std.algorithm: filter, map, each;
import std.file: SpanMode, dirEntries, DirEntry;
import std.stdio: writeln;
import std.array : array;

void main () {
	auto Filesys = ["TEST1/BACKUP", "TEST2/EXPORT"];
	string[][] sizes;
	foreach(FFs; Filesys)
	{
		auto dFiles = dirEntries(FFs, SpanMode.shallow).filter!(a => 
a.isDir).map!(a => a.name);
		foreach (d; dFiles) {
			sizes ~= dirEntries(d, SpanMode.depth).map!(a => a.name).array;
		}
	}
	sizes.each!writeln;
}

It outputs the paths as expected :

["TEST1/BACKUP/FOLDER1/file1", "TEST1/BACKUP/FOLDER1/file2", 
"TEST1/BACKUP/FOLDER1/file3"]
["TEST1/BACKUP/FOLDER2/b1", "TEST1/BACKUP/FOLDER2/b2"]
["TEST2/EXPORT/FOLDER1/file2_3", "TEST2/EXPORT/FOLDER1/file2_1", 
"TEST2/EXPORT/FOLDER1/file2_2"]
["TEST2/EXPORT/FOLDER2/export2", "TEST2/EXPORT/FOLDER2/export3", 
"TEST2/EXPORT/FOLDER2/export1", "TEST2/EXPORT/FOLDER2/export4"]