s***@t-online.de
2012-12-20 18:16:54 UTC
Hello ALl,
I am trying to optimize a load from ( actually a dbimport)
Initially all unlogged and after each test dbspaces are dropped and recreated
they are on a ext2 filesystem. Direct IO switched on.
Machine is on debian linux vmware 4 cpus 16 GB memory.
Configured are 3.000.000 buffers lru cleaning switched off ( 99% and 98 % am doing my own checkpointing) when i run a load from insert into table_where_rowsize_1924_bytes. Columns in the table are char, int, decimal , date types nothing really special.
the database dirtys pages at a rate of 5000 pages per second that is 10 MB a second. top says one oninit is eating aprox 95%CPU dbimport (or dbaccess)
is eating 70 % of cpu.(have tryed it with a german locale and the default, no difference)
This does not seem right.
When i instead copy a table from dbspace a to b it dirtys 100.000 pages or more a second.
When it is time to write the data to disk the i/o subsystem is doing
64 MB a second at 2000 request (iostat says so) and the io subsystem is 100 % busy. In this case writing the data to disk seems to be 6 times faster then
loading it into the buffer cache.
Last but not least writing seems to be done using 32 KB request eq the old 16 pages as MAXIO size. When i add a dbspace using onspaces it does a lot bigger requests and is capable of writing 250 MB a second.
I guess one should take a real close look to this since disks are getting faster
and this is killing performance. so if i can ask for a feature request:
get rid of MAXIO eq 16 pages, make it bigger so we can have a better troughput.
Thanks
Superboer.
I am trying to optimize a load from ( actually a dbimport)
Initially all unlogged and after each test dbspaces are dropped and recreated
they are on a ext2 filesystem. Direct IO switched on.
Machine is on debian linux vmware 4 cpus 16 GB memory.
Configured are 3.000.000 buffers lru cleaning switched off ( 99% and 98 % am doing my own checkpointing) when i run a load from insert into table_where_rowsize_1924_bytes. Columns in the table are char, int, decimal , date types nothing really special.
the database dirtys pages at a rate of 5000 pages per second that is 10 MB a second. top says one oninit is eating aprox 95%CPU dbimport (or dbaccess)
is eating 70 % of cpu.(have tryed it with a german locale and the default, no difference)
This does not seem right.
When i instead copy a table from dbspace a to b it dirtys 100.000 pages or more a second.
When it is time to write the data to disk the i/o subsystem is doing
64 MB a second at 2000 request (iostat says so) and the io subsystem is 100 % busy. In this case writing the data to disk seems to be 6 times faster then
loading it into the buffer cache.
Last but not least writing seems to be done using 32 KB request eq the old 16 pages as MAXIO size. When i add a dbspace using onspaces it does a lot bigger requests and is capable of writing 250 MB a second.
I guess one should take a real close look to this since disks are getting faster
and this is killing performance. so if i can ask for a feature request:
get rid of MAXIO eq 16 pages, make it bigger so we can have a better troughput.
Thanks
Superboer.