Container Array

[Vino.B via Digitalmars-d-learn]

On Friday, 8 September 2017 at 12:14:46 UTC, Vino.B wrote:
> On Friday, 8 September 2017 at 09:51:38 UTC, Ali Çehreli wrote:
>> On 09/07/2017 11:21 PM, Vino.B wrote:
>>
>> >   At last was able to print the output, but i am getting some
>> > "Deprecation" warnings like below and also can you help me in
>> formating
>> > the output to display ulong.
>> >
>> > Output:
>> > Size.d(9): Deprecation: std.container.array.RangeT(A) is not
>> visible
>> > from module Size
>>
>> That's due to std.container.array.RangeT being private. The 
>> deprecation warning is about a bug that leaked such private 
>> symbols when they were imported selectively (I think). Now the 
>> bug is fixed, you won't be able to access the symbol at the 
>> end of the deprecation period.
>>
>> > import std.algorithm: filter, map, fold;
>> > import std.container;
>> > import std.file: SpanMode, dirEntries, isDir;
>> > import std.stdio: File, writefln, writeln;
>> > import std.typecons: tuple, Tuple;
>> > import std.parallelism: parallel;
>> > import std.conv;
>> > import std.range;
>> > Tuple!(RangeT!(Array!(Tuple!string)), RangeT!(Array!ulong))
>> > coSizeDirList () {
>> >     string FFs = "C:\\Temp\\sapnas2\\BACKUP";
>> >     int SizeDir = 1;
>> >     ulong subdirTotal;
>> >     ulong subdirTotalGB;
>> >     Array!(ulong) Subdata;
>> >     auto dFiles = Array!(Tuple!(string)) (dirEntries(FFs,
>> > SpanMode.shallow).filter!(a => a.isDir).map!(a =>
>> tuple(a.name)));
>> >     foreach (d; dFiles[]) {
>> >                 auto SdFiles = Array!(Tuple!(ulong))
>> (dirEntries(d[0],
>> > SpanMode.depth).map!(a => tuple(a.size)));
>> >                 foreach(f; SdFiles[]) { subdirTotal +=
>> f.fold!((a, b) =>
>> > a + b); }
>> >                     subdirTotalGB = (subdirTotal/1024/1024);
>> >                     if (subdirTotalGB > SizeDir) { Subdata ~=
>> > subdirTotalGB; }
>> >                     subdirTotal = 0;
>> >             }
>> >             return tuple (dFiles[], Subdata[]);
>> > }
>> >
>> > void main () {
>> >     writeln(coSizeDirList[]);
>> >     //writefln("%(%-(%-63s %)\n%)", coSizeDirList[]);
>> > }
>>
>> I apologize for not really having time to look at what you're 
>> trying to achieve. I gave you advice which ended up trying to 
>> solve compilation errors.
>>
>> I think the main problem here is to determine directories 
>> above a certain size. So, I think Array should enter the 
>> picture only if built-in arrays are not usable for some 
>> reason. Even better, one should stay with lazy range 
>> algorithms as long as it's possible.
>>
>> How about the following approach, which you can either use 
>> directly or populate an Array with:
>>
>> import std.algorithm: filter, map, sum;
>> import std.file: SpanMode, dirEntries, isDir, DirEntry;
>> import std.stdio: writeln;
>>
>> auto dFiles(string dirName) {
>>     return dirEntries(dirName, SpanMode.shallow).filter!(a => 
>> a.isDir);
>> }
>>
>> auto totalSize(DirEntry dir) {
>>     return dirEntries(dir, SpanMode.depth).map!(a => 
>> a.size).sum;
>> }
>>
>> struct DirInfo {
>>     string name;
>>     ulong size;
>> }
>>
>> auto coSizeDirList (string dirName, ulong sizeLimit) {
>>     return dFiles(dirName).map!(dir => DirInfo(dir.name, 
>> dir.totalSize)).filter!(info => info.size > sizeLimit);
>> }
>>
>> void main () {
>>     writeln(coSizeDirList("./deleteme", 1));
>>
>>     // Only if Array is really needed:
>>     import std.container : Array;
>>     auto arr = Array!DirInfo(coSizeDirList("./deleteme", 42));
>>     writeln(arr[]);
>> }
>>
>> Ali
>
> Hi Ali,
>
>   As stated earlier my release 1 code are still using 
> std.array, so now in release 2 i am converting all my standard 
> array to container array. My program has 5 function and I was 
> able to adopt 4 function to container array, and facing issue 
> with this 1 function. I would like to have all my function to 
> be neither in standard array nor in container array, more over 
> I am facing gc issue in standard array and this is the main 
> reason I would like to convert my function to container array, 
> as this function would find the size of folder which are 
> greater then a specified size in a 5 file system each with 10 
> TB, so i have added the thread(with Local storage) and 
> parallelism to my function as container array give's me the 
> option of reserving memory, so that i would bump it up as and 
> when it is required with any gc issues. All help to resolve 
> this issue would be appreciated.
>
> Updated Code:
> import std.algorithm: filter, map, fold;
> import std.container;
> import std.file: SpanMode, dirEntries, isDir;
> import std.stdio: File, writefln, writeln;
> import std.typecons: tuple, Tuple;
> import std.parallelism: parallel;
> import std.conv;
> import std.range;
>
> Tuple!(RangeT!(Array!string), RangeT!(Array!ulong)) 
> coSizeDirList () {
> 	string FFs = "C:\\Temp\\sapnas2\\BACKUP";
> 	int SizeDir = 1;
> 	ulong subdirTotal;
> 	ulong subdirTotalGB;
> 	Array!(string) Subdir;
> 	Array!(ulong) Subsize;
> 	Tuple!((Array!string), (Array!string)) Result;
> 	auto dFiles = Array!string ((dirEntries(FFs, 
> SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
>     foreach (d; dFiles[]) {
> 				auto SdFiles = Array!ulong ((dirEntries(d, 
> SpanMode.depth)).map!(a => a.size));
> 				foreach(f; SdFiles[]) { subdirTotal += f; }
> 					subdirTotalGB = (subdirTotal/1024/1024); { Subdir ~= d; 
> Subsize ~= subdirTotalGB; }
> 					if (subdirTotalGB > SizeDir)
> 					subdirTotal = 0;
> 		    }
> 			return tuple (Subdir[], Subsize[]);
> }
>
> void main () {
> 	writeln(coSizeDirList[]);
> }
>
> From,
> Vino.B

Hi Ali,

  Was able to resolve the above issue but not sure whether it is 
correct and now i am getting the output as below, request your 
help.

Output:
C:\Temp\sapnas2\BACKUP\dir1, 34, C:\Temp\sapnas2\BACKUP\DND3, 1, 
C:\Temp\sapnas2\BACKUP\DND5, 5

Required Output:
C:\Temp\sapnas2\BACKUP\dir1                34
C:\Temp\sapnas2\BACKUP\DND3                 1
C:\Temp\sapnas2\BACKUP\DND5                 5

Program:
import std.algorithm: filter, map, fold;
import std.container;
import std.file: SpanMode, dirEntries, isDir, isFile;
import std.stdio: File, writefln, writeln;
import std.typecons: tuple, Tuple;
import std.parallelism: parallel;
import std.conv;
import std.range;

Array!string coSizeDirList () {
	string FFs = "C:\\Temp\\sapnas2\\BACKUP";
	int SizeDir = 1;
	ulong subdirTotal;
	ulong subdirTotalGB;
	Array!(string) Subsize;
	Array!string Result;
	auto dFiles = Array!string ((dirEntries(FFs, 
SpanMode.shallow).filter!(a => a.isDir))[].map!(a => a.name));
     foreach (d; dFiles[]) {
				auto SdFiles = Array!ulong(dirEntries(d, 
SpanMode.depth).map!(a => a.size));
				foreach(f; SdFiles[]) { subdirTotal += f; }
				subdirTotalGB = (subdirTotal/1024/1024); { Result ~= d; 
Result ~= to!string(subdirTotalGB); }
				if (subdirTotalGB > SizeDir)
				subdirTotal = 0;
		    }
			return Result;
}

void main () {
writefln("%-(%s, %)", coSizeDirList[]);
}

From,
Vino.B