Yahoo Finance Scraper

[Selim]

On Friday, 12 June 2020 at 19:10:06 UTC, jmh530 wrote:

> Why do you use a class for YahooMinerD (also the name 
> YahooFinanceD might get more people to use)? It doesn't look 
> like you are using any inheritance. I don't see any reason not 
> to change to a struct and avoid new.

Thanks! I liked the new name and updated it. Yeah, I realized 
that about classes and converted to struct. I didn't like the new 
there neither.


> It looks like you have a lot of writeln statements. While these 
> could be helpful, they will also prevent those functions from 
> ever being @nogc. You could use an approach like below and give 
> the user the opportunity to avoid the writelns and allow for 
> attribute inference elsewhere.

I implemented a framework quite similar to the one you described. 
I actually didn't realize how robust templates were in D until 
your comment. Logging stuff is now optional. My application code 
and test code have examples. I don't think MineImpl function can 
be @nogc at this point but please correct me if I'm wrong.


> It looks like the primary way to get the data is from the 
> WriteToJson, correct? What if you want to use the data without 
> writing the JSON to file? For instance, I want to get the data 
> and put it in my own database, or I just want to get the data, 
> do some calculations, and then not save it. There should be a 
> way to get the data out of there without writing to file. 
> std.json can be used for parsing the JSON and there are other 
> libraries out there.

That's true. I added a data frame struct to save all corporate 
actions and prices. It can be accessed from the script. Should be 
quite easy to convert that into a csv or something else too.


> The WriteToJSON method should also allow writing events or 
> prices without needing to write both. You might consider adding 
> the ability to control the frequency (instead of just daily).

Added both of them!

S