Monday, April 30, 2007

Thursday, April 19, 2007

Trouble with commits, Oracle Asynchronous Commit Solution

I'm doing some work loading SHP files into oracle using the oracle example the Oracle Java Shapefile Converter which contains the source code

It all works quite nicely, but i wanted to give some feedback in the tool I'm writing which automates the loading of SHP file datasets into Oracle spatial, with some neat handling along the way...

Anyway, java ain't my first language, but i got working and added some support for writing progress updates back against the table which tracks the source files to be loaded...

It all worked pretty nicely until it hit some large files and started to choke on oracle waits while it wrote the transactions to disk, which was causing waits, which were compounded by my progress being updated every 10s.... everything slowed down to a crawl

The solution was to use "COMMIT WORK WRITE BATCH NOWAIT" feature in 10g, which lets oracle decide when to commit to disk and get the LGWR process to process the redo logs. Works a treat :)

You have to be careful with this because it means stuff isn't being committed to disk straight away, but you can read the result from other sessions, which was my goal.

I have a coldfusion page which polls the table containing the progress updates and calculates the estimated time, throughput per minute and shows a nice CSS progress bar.

My next step i was to use the same approach for the commits for the spatial inserts at the same time. Previously i was am doing a real commit every 10,000 rows....

The last run was done in 41 mins for 1,123,000 rows, the new approach is looking like about 28 minutes to load... so that's about 25% faster, and less flashing HDD lights :)

As a side note, i like developing on a laptop, because I have only a 5400 rpm hard disk and you get to see issues you might not see on a desktop with a faster HD...

I am loading from an external harddisk to avoid I/O contention, BTW

Thursday, April 12, 2007

Keating the song!

the keating song :) true gold

saw this one on crikey today, last day of my subscription..

i am heading os soon, but given it's an election year in australia, i think i am going to renew, it just is simply a much more interesting media outlet than any other australian

interesting to hear the unemployment figure is at a new 30 year low, of course, that's relative to what your measuring.... in australia someone who works 1 hour a week should is counted as being employed, there are a lot of people working part time. then to hear the unemployment figures are likely to put pressure on interest rates, it's sad to see howard's statistical theatrics causing yet even more harm to middle australia...

Tuesday, April 10, 2007

Fullerscreen and the Vines

I guess not everyone knows that F11 in firefox and IE give you a full screen view

Daniel Glazman has created an addon for firefox called
Fullerscreen which makes it truly full screen...

once you have installed it, go to the vines website for a truly great visual experience

John Howard can only do one thing at a time?

PM yet to see $7b national reform plan

Mr Howard says it is odd that Mr Bracks is asking the Commonwealth for extra money when he is yet to sign up to the Government's $10 billion water plan.

does anyone find this a just a little bit ironic or stupid?

Howard comes up with some policy on the back of a handkerchief, figures it's worth $10 billion without asking treasury and then complains Victoria won't sign up, when his own constituency the VFF ( Victorian Farmers Federation ) refuse to agree with it

and then says effectively that it's impossible for him to look at any other policy until this one is signed off? It does explain why he has done so little reform in 10 years doing a boom cycle....

I guess not every politician works the way you do Mr Howard....

Or is it just another case of being caught out doing nothing?

The problem with the federal state government relationship is all Howard's doing, if you work with people, rather than sticking something down their throat and saying take it or leave it, people tend to respond better

maybe it's all getting to much for him?

I like the consensus style of Mr Rudd... sure sometimes you need to take the lead aggressively, but that's often better as a last resort, not a first step

anyway, I waiting for the $300 million dollar advertising campaign from the government before the next election

Wednesday, April 04, 2007

Great XP Disk Imaging Software

Ok, I got my laptop back and restored my hard drive from a backup image.

Image for Windows rocks..... nice and easy to install and use, and it's simple enough that anyone can use it....

Make sure you install phylock (it allows you to backup while windows is running) and make a linux restore cd as it gives you a few more options for the geeks out there to play with...

Damn fine simple stuff....

I went looking for open source stuff, but it was all to complex, it's a simple task

SQL Server 2005 Notes

Ok, new project and i need to play with SQL Server 2005...

I remember what a cow Oracle was to install back in the 90's, with Oracle 10g they finally got the install right.. it just works, presents all the options in a logical fashion and it's quick...

SQL Server 2005? a typical install is easy, custom is a silly complex MS experience.

How hard is it to give the option of installing everything under one directory? Plus the silly install options.. make available options and the like... arghhhh

anyway got it up and running now.....

if you import an old database an try to do a diagram and get this error

"Database diagram support objects cannot be installed because this database does not have a valid owner. To continue, first use the Files page of the Database Properties dialog box or the ALTER AUTHORIZATION statement to set the database owner to a valid login, then add the database diagram support objects. "

(why can't they have a simple error code a la oracle? and i got an error trying to copy the text to the clipboard, sigh)

try this solution

Sunday, April 01, 2007

Is DIGITAL SBS skipping on your DigiCrystal ?

I have a DigiCrystal Set Top Box SDT-8890 which is working well except SBS DIGITAL skips now (in melbourne) and then
googling round i found this tip, Tune Ch 29 to 536.7 instead of 536.5, which solves the problem.