mysql-native: SQL Transaction support?
salvari via Digitalmars-d-learn
digitalmars-d-learn at puremagic.com
Fri Sep 18 06:46:28 PDT 2015
On Thursday, 17 September 2015 at 21:19:07 UTC, Gary Willoughby
wrote:
> On Thursday, 17 September 2015 at 19:47:33 UTC, salvari wrote:
>> I'm parsing a text input file, the generated sql is about 1
>> million lines of SQL. By using mysql-native it takes about 4
>> hours to load data.
>
> I've used mysql-native before to handle hundreds of millions of
> rows of data and I remember it being blazingly fast, nowhere
> near the time you suggest. Perhaps try to split the SQL into
> single lines and use prepared statements, this is where the big
> wins in speed are.
>
> Expecting any server to deal with a million lines of SQL in a
> timely manner is asking too much.
On Thursday, 17 September 2015 at 21:19:07 UTC, Gary Willoughby
wrote:
> On Thursday, 17 September 2015 at 19:47:33 UTC, salvari wrote:
>> I'm parsing a text input file, the generated sql is about 1
>> million lines of SQL. By using mysql-native it takes about 4
>> hours to load data.
>
> I've used mysql-native before to handle hundreds of millions of
> rows of data and I remember it being blazingly fast, nowhere
> near the time you suggest. Perhaps try to split the SQL into
> single lines and use prepared statements, this is where the big
> wins in speed are.
>
> Expecting any server to deal with a million lines of SQL in a
> timely manner is asking too much.
Thanks for the advice.
I know I should have used prepared inserts but the input file is
a bit messy and I was in a hurry.
MySQL server is not the problem, as I said, executing exactly the
same SQL divided in chunks and using transaction gets the job
done in 1 minute instead of 4 hours. So far my problem is solved
but I would like to know more about transaction support in
mysql-native library.
I have to make more tests.
--
salvari
More information about the Digitalmars-d-learn
mailing list