dlang-requests 1.1.0 released

[ikod]

On Sunday, 5 April 2020 at 11:53:29 UTC, Petar Kirov [ZombineDev] 
wrote:
> On Sunday, 5 April 2020 at 08:59:50 UTC, ikod wrote:
>> Hello!
>>
>> Just a note that dlang-requests ver 1.1.0 released with new 
>> 'ByLine' interfaces added for get/post/put requests.
>>
>> range algorithms can be applied to server responses, so that 
>> simple chain
>>
>> getContentByLine("https://httpbin.org/anything")
>>     .map!"cast(string)a"
>>     .filter!(a => a.canFind("data"))
>>
>> should work.
>>
>> These calls work lazily so you can apply them to large 
>> documents.
>>
>> dlang-requests - HTTP client library, inspired by 
>> python-requests with goals:
>>
>> small memory footprint
>> performance
>> simple, high level API
>> native D implementation
>>
>> https://github.com/ikod/dlang-requests
>> https://code.dlang.org/packages/requests
>>
>> Always waiting for your bugreports and proposals on project 
>> page.
>>
>> Best regards!
>
> Nice work!
>
> One quick suggestion: avoid direct casting from `ubyte[]` to 
> `string`:
>
> /+dub.sdl:
> dependency "requests" version="~>1.1"
> +/
>
> void main()
> {
>     import requests : getContentByLine;
>     import std : assumeUTF, canFind, each, filter, map, write;
>
>     getContentByLine("https://httpbin.org/anything")
>         .map!assumeUTF // instead of map!"cast(string)a"
>         .filter!(a => a.canFind("data"))
>         .each!write;
> }
>
> 1. From a code-style point of view, assumeUTF is better as it 
> shows the intention to the reader - assume that the content is 
> valid UTF8 encoded text, without performing validation. And if 
> there are UTF8 errors, it is easy to go back and add validation 
> there.
> 2. Avoid casting mutable data to immutable. The data path in 
> your library is rather complex (getContentByLine -> _LineReader 
> -> LineSplitter -> Buffer -> ...) and so it was hard to 
> understand from a quick glance whether or not the buffer array 
> is reused (but I would guess that it is). If the buffer array 
> is reused, it means that the result of calling 
> _LineReader.front() may be modified at a later point in time, 
> which I think is obvious that it can lead to some rather nasty 
> bugs in users' code.

Internally my code do not modify these data, but I understand 
your concern.

>
> I suggest you look into Steven's iopipe[1] library, as I 
> believe it can help you clean up and refactor this part of the 
> codebase (and can probably yield some performance improvements 
> along the way).
>
> [1]: https://github.com/schveiguy/iopipe

Thanks! You are totally right regarding casting mutable to 
immutable. Base code was written when I did not understand this 
problem well enough. I know how to fix this but this probably 
require large enough rewrite. I hope I'll do this in some near 
future.

PS. I also have package which convert nogc unique mutable byte 
array to immutable byte array and attach this immutable data "as 
is" to buffer (which is essentally array of immutable(ubyte)[]). 
Then every operation on buffer (trim, split, search, etc) 
performed on immutable with zero copy and with warranties on data 
immutability.