D-styled data file

[Georg Wrede]

Saaa wrote:
> Simply put, I'd like a D styled dataformat alike JSON.
> Somehow saying that I would like it D styled makes it difficult to 
> understand :)
> 
> "Georg Wrede" <georg.wrede at iki.fi> wrote in message 
> news:gta2r9$1vls$1 at digitalmars.com...
>> Saaa wrote:
>>>>> I'm still dangling between a full parsing at load and parse on demand.
>>>>> A parse on demand (get) would do the following steps:
>>>>> Get the type of the variable in which the data must be stored (in 
>>>>> string format)
>>>>> Search for this type in every line of the char[][], when found check 
>>>>> whether the name is the same
>>>>> and then convert the chars to that type and place the data in the 
>>>>> variable.
>>>> This sounds complicated. Can you decide the input file format? Or is it 
>>>> from some other program that you can't change?
>>> The input file format would be like D
>>> Like file.dat from the original post.
>>> I just often have the need to save large arrays and other variables and I 
>>> thought
>>> why not just save them like the way I would use them in my modules.
>> Err, ok, lemme assume this is kind-of like a scrapbook thing. You collect 
>> nice arrays, filled data structures and the like. And the you'd want to 
>> read them into your programs.
> Yes, except that the program creating them is the same as the one loading 
> them.
> It's a simple safe (with very large multi arrays)

Well, then were really talking about Classic Serialzation. In other 
words, a program writes data somewhere (possibly in human readable 
format), and then later (as in tomorrow) reads it in.

>> Now, to know how to use the stuff, the program would need to not only be 
>> able to handle the data structures, but also know their names. Sure, you 
>> could have a parser that returns first the name of the thing, and then the 
>> thing itself. But then, to use it, you probably couldn't have an Universal 
>> program. Rather, you'd have to write the program different for each time 
>> you decide to use a particular item from the scrap book.
>>
> The JSON parser does the same thing. You give it the name of the array you 
> want to read.
> 
> This is what the corresponding data files look like
> - JSON version
> "array" :
> [
> [
> [
> 1,
> 2,
> ],
> [
> 3,
> 4,
> ]
> ]
> [
> ... etc :)
> --
> 
> -D styled version
> int[2][2][2] array = [[
> [1,2],
> [3,4]],
> [
> [5,6],
> [7,8]];

So, are we talking basically saving the state of the program for the 
next run?

>> Which actually leads us to another simple solution. Why not write your 
>> scrapbook simply in total D format, and then include it in the program?
> I really can't ask my customers to rebuild their app everytime they save 
> something :)
> 
>>>> Suppose you already had this get function. How would you use it? An 
>>>> example would help.
>>> I use the get functions a few posts back, in the main:
>>>
>>> ---
>>> void main()
>>> {
>>>  char[] filename = `data.dat`;
>>>  char[][] file;
>>>
>>>  try{
>>>   file = splitlines( cast(char[])read(filename) );
>>>  }
>>>  catch{
>>>   throw new Exception("Couldn't load : " ~ filename);
>>>  }
>>>
>>>  DData file_dd = new DData(file);
>>>  int i;
>>>  file_dd.get(`i`, i);
>>> }
>>> ---
>>>
>>> The data.dat should have a line like this to work:
>>> --- data.dat
>>> int i = 10;
>>> ---
>> So here main has to know that there will be a variable i in the data file, 
>> then get it, then presumably assign the value to the here defined "int 
>> i;". This way you have to do everything (and more) you would if you were 
>> to rename data.dat to data.d, include it, and then just use the variables.
> The get function doesn't need to know that there will be an i in the data,
> it will just search for it.

Exactly. The get function doesn't have to know. But the rest of the 
program will have to know what to do with what get gives it. Right? 
Actually, /that/ is what I'd see as the hard part here.

>>> I hope it all makes sense :)
>> So, either I still don't get it, or you're doing Double Thinking. A 
>> picture might clarify:
>>
>> http://ocw.mit.edu/NR/rdonlyres/A166EB14-E7FD-473F-98FB-741A9817540C/0/chp_triangle.jpg
>>
>> It's very often that programmers spend time attempting to do something 
>> that is possible, but when it comes down to it, it either isn't, or it 
>> takes an inordinate amount of work /compared/ to the value of the goal.
>>
> I know it is possible because
> 1. it isn't much different than the JSON parser from tango
> 2. the compiler does the same
> 
> If the JSON writer won't write every item (which will be thousands of items)
> on a new line and reading in multi arrays, then I think I need to witch to 
> Tango :D 

No problem. I'm not here to "save you from Tango". :-)


Okay, (I still feel I don't get it entirely), now I assume the thing is 
to save the state for the next invocation tomorrow.

Then I'd assume, you've basically got two things to do. First, write the 
code to save basic data types, and then the code to save compound types 
(as in structs, classes, trees, etc.). An OO approach would be to have 
each class have methods to write and to read the data.

I can't help feeling that either using e.g. the Json parser, or writing 
your own IO routines, should be the easy part. But then I feel using 
this get function would be the harder thing.

Incidentally, Andrei is about to publish printing arbitrary data 
structures in D (as part of Phobos), and if I suspect correctly, this 
will also be done in a way that you can both write /and/ read stuff from 
files. A little like Json. But of course, this might not be soon enough 
for any real program for a customer.