Web crawler/scraping
Adam D. Ruppe
destructionator at gmail.com
Wed Feb 17 13:13:00 UTC 2021
On Wednesday, 17 February 2021 at 12:12:56 UTC, Carlos Cabral
wrote:
> I'm trying to collect some json data from a website/admin panel
> automatically, which is behind a login form.
Does the website need javascript?
If not, my dom.d may be able to help. It can download some HTML,
parse it, fill in forms, then my http2.d submits it (I never
implemented Form.submit in dom.d but it is pretty easy to make
with other functions that are implemented, heck maybe I'll
implement it now if it sounds like it might work).
Or if it is all json you might be able to just craft some
requests with my lib or even phobos' std.net.curl that submits
the login request, saves a cookie, then fetches some json stuff.
I literally just rolled out of bed but in an hour or two I can
come back and make some example code for you if this sounds
plausible.
More information about the Digitalmars-d-learn
mailing list