vibe.d: How to get the conent of a file upload ?

[Steven Schveighoffer]

On 9/17/20 8:07 PM, wjoe wrote:
> 
> Not a reply to this post in particular but to all the ones I've read so 
> far.
> 
> If I understand correctly. Vibe parses the form data and writes all 
> files to disk. Where to ?

See the code here: 
https://github.com/vibe-d/vibe.d/blob/ebebfa827f568cc9bced4bec2b66edc043a8adf7/inet/vibe/inet/webform.d#L311

> Can I configure it ? I don't want libraries to just write data to my 
> file systems without me setting this up. Nowhere did I find this 
> behavior described in the docs.

No, not at the moment. Which is why I was saying, it could be an 
enhancement request to vibe.

> And if not, how is data processed with a 10mb file upload followed by a 
> few number fields ?

All the data is processed before the accessor to the form data or the 
file data. It HAS to be this way, as the data is still on the incoming 
network socket.

> It needs to read all of the file data to get to the other data fields, 
> doesn't it ?

Yes

> 
> I'm sorry this is completely counter intuitive. I can understand the 
> memory/security risks and all but I have no intention to hack, DOS or 
> however else disrupt my private server in my private network with 
> garbage data. I just want to get the data in a byte[].

Again, enhancement request needed. The code currently is hard-coded to 
write to disk.

> 
> Why does the lib not simply reject files that are unreasonably 
> (configurable) big ?

If you had 1000 requests being processed simultaneously, and each of 
those requests provided 10MB files, then you now need potentially 10GB 
of RAM to hold those requests. This doesn't scale when the application 
is unknown to vibe.

But again, solved with an enhancement that allows you to process the 
data in your code. I'll file the enhancement request for you, as I think 
it's a nice addition.

> Writing files to disk in order to then needing to copy them somewhere 
> else or to read them back into memory for further processing sounds, 
> above all else, incredibly inefficient.

Agreed. In my case, it was an actual copy, as the location of the stored 
data was on a different filesystem than the temporary files.

> I might not even want to keep the file and drop it.

Yep, it's a waste in that case.

> 
> I guess it's no problem to parse the data myself, but then what's the 
> point in using a framework ?

Agreed.

> Are there other frameworks besides vibe that can do what I want ?

In D, I'm not sure what the other frameworks do. I believe there are 
others if you search on code.dlang.org, you should be able to find some.

> I'm sorry for the rant, developing this kind of software is a pain in 
> the drain and stresses me out to no end. It sucked hard in the past with 
> php and decades later with python, ruby, D, you name it it still sucks ;)

web development sucks in general ;) Yet, we all still do it.

-Steve