Reading web pages

[Xan xan]

Thank you very much. I should invite you to a beer ;-)

For the other hand,

I get this error:

[Excepció: std.conv.ConvException@/usr/include/d2/4.6/std/conv.d(1640):
Can't convert value `HTT' of type string to type uint]


if I only want the length:

//D 2.0
//gdmd-4.6 <fitxer> dhttpclient => surt el fitxer amb el mateix nom i .o
//Usa https://github.com/Bystroushaak/DHTTPClient
//versió 0.0.2
import std.stdio, std.string, std.conv, std.stream;
import std.socket, std.socketstream;
import dhttpclient;

int main(string [] args)
{
    if (args.length < 2) {
		writeln("Usage:");
		writeln("   ./spider {<url1>, <url2>, ...}");
		return 0;
	}
	else {
		try {
			string[string] capcalera = dhttpclient.FFHeaders;
			HTTPClient navegador = new HTTPClient();
			navegador.setClientHeaders(capcalera);

			foreach (a; args[1..$]) {
				auto tamany = cast(ubyte[]) navegador.get(a);
				writeln("[Contingut: ", tamany.length, "]");
			}
		}
		catch (Exception e) {
			writeln("[Excepció: ", e, "]");
		}
		return 0;
	}
}


In theory, tamany.length is completely defined.

Xan.

2012/1/20 Bystroushaak <bystrousak at kitakitsune.org>:
> rawWrite():
>
> stdout.rawWrite(cast(ubyte[]) navegador.get(a));
>
>
> On 20.1.2012 18:18, Xan xan wrote:
>>
>> Mmmm... I understand it. But is there any way of circumvent it?
>> Perhaps I could write to one file, isn't?
>>
>>
>>
>> 2012/1/20 Bystroushaak<bystrousak at kitakitsune.org>:
>>>
>>> Thats because you are trying writeln binary data, and that is impossible,
>>> because writeln IMHO checks UTF8 validity.
>>>
>>>
>>> On 20.1.2012 18:08, Xan xan wrote:
>>>>
>>>>
>>>> Before and now, I get this error:
>>>>
>>>> $ ./spider http://static.arxiv.org/pdf/1109.4897.pdf
>>>> [Excepció: std.conv.ConvException@/usr/include/d2/4.6/std/conv.d(1640):
>>>> Can't convert value `HTT' of type string to type uint]
>>>>
>>>> The code:
>>>>
>>>> //D 2.0
>>>> //gdmd-4.6<fitxer>    =>    surt el fitxer amb el mateix nom i .o
>>>> //Usa https://github.com/Bystroushaak/DHTTPClient
>>>> import std.stdio, std.string, std.conv, std.stream;
>>>> import std.socket, std.socketstream;
>>>> import dhttpclient;
>>>>
>>>> int main(string [] args)
>>>> {
>>>>     if (args.length<    2) {
>>>>                writeln("Usage:");
>>>>                writeln("   ./spider {<url1>,<url2>, ...}");
>>>>                return 0;
>>>>        }
>>>>        else {
>>>>                try {
>>>>                        string[string] capcalera = dhttpclient.FFHeaders;
>>>>                        //capcalera["User-Agent"] = "arachnida yottiuma";
>>>>                        HTTPClient navegador = new HTTPClient();
>>>>                        navegador.setClientHeaders(capcalera);
>>>>
>>>>                        foreach (a; args[1..$]) {
>>>>                                writeln("[Contingut: ", cast(ubyte[])
>>>> navegador.get(a), "]");
>>>>                        }
>>>>                }
>>>>                catch (Exception e) {
>>>>                        writeln("[Excepció: ", e, "]");
>>>>                }
>>>>                return 0;
>>>>        }
>>>> }
>>>>
>>>>
>>>>
>>>> What happens?
>>>>
>>>>
>>>> 2012/1/20 Bystroushaak<bystrousak at kitakitsune.org>:
>>>>>
>>>>>
>>>>> It is unlimited, you just have to cast output to ubyte[]:
>>>>>
>>>>> std.file.write("logo3w.png", cast(ubyte[])
>>>>> cl.get("http://www.google.cz/images/srpr/logo3w.png"));
>>>>>
>>>>>
>>>
>