mysql-native v2.1.0

[Steven Schveighoffer]

On 3/8/18 6:57 AM, bauss wrote:
> On Thursday, 8 March 2018 at 06:49:52 UTC, Nick Sabalausky (Abscissa) 
> wrote:
>> On 03/07/2018 02:32 PM, bauss wrote:
>>>
>>> Wait why has it been updated to array() ? So it's not a real range 
>>> anymore? Or was it always represented as an array behind the scenes?
>>>
>>> I just feel like allocating it into an additional array is a waste of 
>>> memory? But if it was always like that I guess it doesn't matter.
>>>
>>
>> query() returns an input range. You can only access one element at a 
>> time (as its read from the network) and you don't know how many there 
>> are ahead of time, BUT it avoids allocating a whole array to store 
>> everything.
>>
>> In addition to query(), there used to also be a querySet(). The 
>> querySet() would allocate an array and read ALL the results into it so 
>> you could get random-access. But that's exactly what you already get 
>> when you call array() on an input range (such as the input range 
>> returned by query), so querySet was deemed redundant and eliminated.
>>
>> So if you had code that *did* need an array allocated to store all the 
>> results, then "querySet()" has been replaced with "query().array". But 
>> like you said, if you don't really need the array, then there's no 
>> need to call array() and waste the memory.
>>
>>
>>> However idk what I changed, but the issue stopped for me.
>>>
>>> However I still have this issue:
>>>
>>> https://github.com/mysql-d/mysql-native/issues/153
>>>
>>> (Currently trying to see if I can make a minimal example, but it's 
>>> kinda hard to make a minimal example since it's from my Diamond MVC 
>>> (vibe.d) library and it isn't used until deep nesting of the 
>>> application.
>>>
>>> Anyway before I report anything else I could easily be doing 
>>> something wrong. There hasn't exactly been any good examples on how 
>>> to use it with vibe.d so it has pretty much been a trial and error 
>>> thing for me.
>>
>> Using mysql-native with vibe.d isn't any different from using it 
>> without vibe.d.
>>
>> It's recommended to use MySQLPool to make a Connection rather than 
>> doing "new Connection" directly simply because connecting is faster 
>> that way (though "new Connection" will still work).
>>
>> But aside from that, there is absolutely nothing different about 
>> mysql-native whether you're using vibe.d or not.
>>
>>
>>> So basically I keep an associative array of connection pools based on 
>>> connection strings like below:
>>>
>>> private static __gshared MySQLPool[string] _pools;
>>>
>>> And then I retrieve a connection with the function below.
>>>
>>> Perhaps I'm not supposed to make a new pool every time, but there is 
>>> someway to retrieve a pool already? Maybe that's what I'm doing wrong?
>>>
>>> private static shared globalPoolLock = new Object;
>>>
>>> private Connection getMySqlConnection(string connectionString)
>>> {
>>>    auto pool = _pools.get(connectionString, null);
>>>
>>>    if (!pool)
>>>    {
>>>      synchronized (globalPoolLock)
>>>      {
>>>        pool = new MySQLPool(connectionString);
>>>
>>>        _pools[connectionString] = pool;
>>>      }
>>>    }
>>>
>>>    return pool.lockConnection();
>>> }
>>>
>>> After I retrieve the connection then it's basically like the code I 
>>> showed you, but that seem to be correct, yes?
>>
>> Does your application need to support multiple connection strings 
>> while it's running? That's pretty rare unless you're making something 
>> like phpMyAdmin (and even then, I'd probably do it a little 
>> differently). Normally you'd just make one connection pool:
>>
>> MySQLPool pool;
>>
>> Then "new" that once with your connection string when you start up, 
>> and you're good.
>>
>> I guess I can imagine some potential use-cases that get more 
>> complicated than that, but that's really up to your own project's needs.
>>
>> > However I still have this issue:
>> >
>> > https://github.com/mysql-d/mysql-native/issues/153
>> >
>> > (Currently trying to see if I can make a minimal example, but
>> it's kinda
>> > hard to make a minimal example since it's from my Diamond MVC
>> (vibe.d)
>> > library and it isn't used until deep nesting of the
>> application.
>>
>> I'm only guessing here, but I wonder if that might be because you seem 
>> to be trying to share pools and connections across threads. I don't 
>> know whether vibe is designed to share TCP connections across threads 
>> or not. I'd say, try ripping out all that 
>> shared/__gshared/synchronized stuff and see how that works.
> 
> But if you can't store the pools anywhere, how are you supposed to use 
> them with vibe.d?
> 
> Creating a new pool for every thread seems expensive and dosn't that 
> defeat the purpose of using pools in the first place?
> 
> I mean a 1000 people could connect to a website and potentially that 
> could create a thousand threads (It probably won't, BUT it could.) and 
> thus it can't be afforded to create a pool per request.

Well, vibe.d shouldn't be doing that. It creates fibers per request, not 
threads, no?

I would expect no more than one thread per core to be created, and you 
can afford that many pools (I'd create them at program/thread startup in 
any case).

The point of a pool is to avoid some costly setup. In my case, I'm not 
even closing the connection because I feel the "cost" of allocating a 
connection from the heap isn't worth worrying about. But I also limit 
the pool so it's only going to allow X concurrent Db connections per thread.

-Steve