Comparison : mysql-native + asdf and hunt-database + asdf

[Vino]

On Saturday, 7 November 2020 at 14:38:35 UTC, Vino wrote:
> On Saturday, 7 November 2020 at 14:16:46 UTC, Vino wrote:
>> On Saturday, 7 November 2020 at 12:29:46 UTC, Andre Pany wrote:
>>> On Friday, 6 November 2020 at 04:58:05 UTC, Vino wrote:
>>>> [...]
>>>
>>> While doing performance measurements you should do each test 
>>> multiple time (at least 5 times). There could be for example 
>>> an effect that executing a db query the first time is a lot 
>>> slower than afterwards.
>>>
>>> Maybe mysql native is slower, but maybe this isn't the case, 
>>> just the way performance measurements was done was 
>>> incorrectly.
>>>
>>> Kind regards
>>> Andre
>>
>> Hi Andre,
>>
>>   The benchmark was performed with 100 executions as below
>>
>> void main()
>> {
>>  void f() {
>>  Array!string[string] data = getdata("TEST");
>>  foreach(i; data) { writeln(i[]); }
>>  }
>>  auto r = benchmark!(f)(100);
>>  Duration t = r[0];
>>  writeln(t);
>> }
>
> Hi All,
>
>   Upon further testing, below are the observations, based on 
> different compliers (DMD/LDC), using LDC the size of the 
> executable is reduced but the run time is increased from 5 to 6 
> sec , the document states that if we use the complier option 
> dub --"build=release --compiler=ldmd2" it would enhance the 
> performance  but as per our analysis we do not see and 
> improvement, below is the runtime
>
> LDC                                      : 6 secs, 136 ms, 97 
> μs, and 3 hnsecs
> LDC(dub --build=release --compiler=ldmd2): 6 secs, 203 ms, 283 
> μs, and 6 hnsecs
>
>
> Observations
> ********************
> Component 	:  mysql-native + asdf	
> Complier	:  DMD
> Size 		:  17MB
> Duration	:  10 secs, 189 ms, 919 μs, and 3 hnsecs
>
> Component 	:  mysql-native + asdf	
> Complier	:  LDC
> Size 		:  20MB
> Duration	:  10 secs, 526 ms, 350 μs, and 6 hnsecs
>
> Component 	:  mysql-native + asdf	
> Complier	:  LDC(dub --build=release --compiler=ldmd2)
> Size 		:  3.7MB
> Duration	:  10 secs, 411 ms, 793 μs, and 8 hnsecs
> *************************************************************
> Component 	:  hunt-database + asdf	
> Complier	:  DMD
> Size 		:  81MB
> Duration	:  5 secs, 916 ms, 418 μs, and 3 hnsecs
>
> Component 	:  hunt-database + asdf	
> Complier	:  LDC
> Size 		:  50MB
> Duration	:  6 secs, 136 ms, 97 μs, and 3 hnsecs
>
> Component 	:  hunt-database + asdf	
> Complier	:  LDC(dub --build=release --compiler=ldmd2)
> Size 		:  17MB
> Duration	:  6 secs, 136 ms, 97 μs, and 3 hnsecs
>
> From,
> Vino.B

Hi All,

   After further analysis we suspect that the issue is at the 
package std.net.curl the flow of the program is as below

hunt.database : queries a single table(12 column's and 3 rows) 
which contains the 3 api url details(getApiconf)

std.net.curl  : call those api's (parallel) and the send the data 
to json parser

asdf          : Parse the output of the api's and store the data 
in a array Array!string[string] data

Code of the API calls
import common.GetApiconf;
import core.stdc.stdlib: exit;
import hunt.database;
import std.algorithm: joiner, sort;
import std.container.array;
import std.conv: to;
import std.net.curl : get, HTTP, CurlOption;
import std.parallelism: parallel;
import std.typecons: Tuple, tuple;

void main ()
{
  Array!(Tuple!(int,string)) apidata;
  Row[] result;
  result = getApiconf("TST");
  foreach(i, k; parallel(result,1))
     {

       string apihost = result[i][0].get!(string);
       int apiport = result[i][1].get!(int);
       string apiuser = result[i][2].get!(string);
       string apipass = result[i][3].get!(string);
       int seq = result[i][4].get!int;
       string apiuri = result[i][5].get!(string);
       int connecttimeout = result[i][6].get!(int);
       int tcp_nodelay = result[i][7].get!(int);
       int http_version = result[i][8].get!(int);
       int sslversion = result[i][9].get!(int);
       int use_ssl = result[i][10].get!(int);
       int ssl_verifypeer = result[i][11].get!(int);

       string url = to!string(([apihost, ":",  to!string(apiport), 
apiuri]).joiner);
       string usrpass = to!string(([apiuser, ":", 
apipass]).joiner);

       auto https = HTTP();
       https.handle.set(CurlOption.userpwd, usrpass);
       https.handle.set(CurlOption.connecttimeout, connecttimeout);
       https.handle.set(CurlOption.tcp_nodelay, tcp_nodelay);
       https.handle.set(CurlOption.http_version, http_version);
       https.handle.set(CurlOption.sslversion,  sslversion);
       https.handle.set(CurlOption.use_ssl,  use_ssl);
       https.handle.set(CurlOption.ssl_verifypeer, ssl_verifypeer);
       https.handle.set(CurlOption.url, url);
       https.method(HTTP.Method.get);
       https.StatusLine st;
       https.onReceiveStatusLine = (https.StatusLine st) { if 
(st.code != 200)
                     {
                      throw new Exception(st.reason);
                     }
       };
       ubyte[] content;
       https.onReceive = (ubyte[] data) {
                                          content ~= data; return 
data.length;
                                        };
       https.perform();
       scope(failure) { https.shutdown; exit(-4); } scope(exit) 
https.shutdown;
       apidata.insert(tuple(seq, cast(string) content));
  }
  writeln(apidata[].sort);
}

From,
Vino.B