SQLite 3.6.23.1 wrapper + connector

[Rory McGuire]

On Tue, 20 Jul 2010 14:23:07 +0200, awishformore <awishformore at gmail.com>  
wrote:

> Am 20.07.2010 08:56, schrieb Rory McGuire:
>> On Tue, 20 Jul 2010 01:37:10 +0200, awishformore
>> <awishformore at gmail.com> wrote:
>>
>>> Am 19.07.2010 21:34, schrieb Rory McGuire:
>>>> On Mon, 19 Jul 2010 20:39:35 +0200, awishformore
>>>> <awishformore at gmail.com> wrote:
>>>>
>>>>> Am 19.07.2010 17:50, schrieb Johannes Pfau:
>>>>>> On 19.07.2010 00:44, awishformore wrote:
>>>>>>> Hello there.
>>>>>>>
>>>>>>> I've converted the .h file of the latest SQLite version to .d and I
>>>>>>> thought I'd let the world know (as suggested on IRC ;). Maybe it  
>>>>>>> will
>>>>>>> save someone some work.
>>>>>>>
>>>>>>> I've also written a nice connector class that wraps all the C
>>>>>>> functions
>>>>>>> for convenient use in your application. It's very basic, but I will
>>>>>>> probably add more features (like automatic preparation of  
>>>>>>> statements
>>>>>>> and
>>>>>>> automatic caching of several prepared statements) later.
>>>>>>>
>>>>>>> For the time being both files are included in the download:
>>>>>>> http://nexunity.com/sqlite4d.rar
>>>>>>>
>>>>>>> I'm pretty new to this kind of stuff, so what I did to get it to  
>>>>>>> work
>>>>>>> was compiling the latest SQLite dll from source with dmc and then
>>>>>>> link
>>>>>>> the .obj file into my app ( http://nexunity.com/sqlite3.obj ).
>>>>>>>
>>>>>>> I'm sure there is a better way like compiling it as static lib (dmc
>>>>>>> complained about no entry point) or having some kind of other file  
>>>>>>> to
>>>>>>> link into your app in order for it to compile and then use the  
>>>>>>> dll. I
>>>>>>> however couldn't figure it out and it works for me. Don't hesitate  
>>>>>>> to
>>>>>>> teach me nonetheless.
>>>>>>>
>>>>>>> Any kind of feedback is always appreciated.
>>>>>>>
>>>>>>> Greetings, Max.
>>>>>> Nice work!
>>>>>>
>>>>>> I'm by no means an expert in d, but I wonder whether the following  
>>>>>> in
>>>>>> the connector could cause troubles:
>>>>>> --------------------------------------
>>>>>> int open(string db_name)
>>>>>> {
>>>>>> return sqlite3_open_v2(cast(char*)db_name,
>>>>>> --------------------------------------
>>>>>>
>>>>>> I always thought just casting to the pointer type isn't really  
>>>>>> safe. I
>>>>>> guess you should use std.strings toStringz.
>>>>>>
>>>>>> (Off topic: std.strings toStringz uses the GC to store the c
>>>>>> string. Now
>>>>>> if I passed the returned pointer to a C function, but didn't store a
>>>>>> reference in the D program, couldn't that c string get collected  
>>>>>> while
>>>>>> the c library would expect it to be still there?)
>>>>>
>>>>> Hello there.
>>>>>
>>>>> I'm not an expert for anything related to programming, but I'm pretty
>>>>> sure the string/char[]/char* isn't needed by SQLite after the  
>>>>> function
>>>>> returns, as you will always be using the sqlite4 object pointer.
>>>>>
>>>>> As for the issue about null-terminated C strings, I was initially
>>>>> expecting this to not work, but tried anyway. I guess this has to do
>>>>> with the fact that SQLite interprets the char* as UTF-8, which is  
>>>>> what
>>>>> D uses. With only a few lines of code, UTF-16 would also work with  
>>>>> the
>>>>> wrapper by the way, but I won't ever use it, so I didn't bother.
>>>>>
>>>>> I added quite a few unit tests to the file in order to verify stuff
>>>>> like this and all of them seem to pass without the slightest issue  
>>>>> and
>>>>> there are no memory leaks either, so I'm assuming that at least the
>>>>> basic functionality of my connector class is absolutely safe to use.
>>>>>
>>>>> The only thing I changed except for the unit tests is the
>>>>> ptrtostr/ptrtoblob to access the pointer like an array instead of a
>>>>> while loop and the else -> else static if(T==sqlite3_value*) to  
>>>>> ensure
>>>>> a valid parameter is always passed.
>>>>>
>>>>> Greetings, Max.
>>>>>
>>>>
>>>> Did you test with a string that was not in the code itself, e.g. from  
>>>> a
>>>> config file?
>>>> String literals are null terminated so you wouldn't have had an issue  
>>>> if
>>>> all your strings were literals.
>>>> Utf8 doesn't contain the string length, so you will run in to problems
>>>> eventually.
>>>>
>>>> You have to use toStringz or your own null terminator. Unless of  
>>>> course
>>>> you know that the function will always be
>>>> taking string literals. But even then leaving something like that up  
>>>> to
>>>> the programmer to remember is not exactly
>>>> fool proof.
>>>>
>>>> Enjoy.
>>>> ~Rory
>>>
>>> Hey again and thanks for the hint. I tried finding something on the DM
>>> page about string literals being null terminated and while the section
>>> about string literals didn't even mention it, it was said some place
>>> else.
>>>
>>> That explains why using string literals works even though I expected
>>> it to fail. It's indeed good to know and adding std.string.toStringz
>>> is probably a good idea ;). Thanks.
>>>
>>> Greetings, Max.
>>
>> sure, I must admit it is annoying when the same code can do different
>> things just because of where the data came
>> from. It would be easier to notice the bug if d never added a null on
>> literals, but then there would also be a lot more
>> usages of toStringz.
>>
>> I think if you want to test it you can do:
>> auto s = "blah";
>> open(s[0..$].dup.ptr); // duplicating it should put it somewhere else
>> // just slicing will not test
>
> When thinking about it, it makes sense to have string literals null  
> terminated in order to have C functions work with them. However, I  
> wonder about some stuff, for instance:
>
> string s = "string";
> // is s == "string\0" now?
> char[] c = cast(char[])s;
> // is c[6] == '\0' now?
> char* p = s.ptr;
> // is *(p+6) == '\0' now?
>
> I think use of the zero terminator should be consistent. Either make  
> every string (and char[] for that matter) zero terminated in the  
> underlying memory for backwards compatibility with C or leave it to the  
> user in all cases.
>
> /Max

perhaps the NULL is there because its there in the executable file?
NULL is also often after a dynamic array simply because of d always  
initializing memory, and
when you get an allocation often a larger amount is allocated which  
remains NULL.