Thursday, November 8, 2007

Effective data slices

Ok. I have not recieved any "Remove me from the list" email. I presume that you read my email &

- said good to know topic
- said let me save this for future reading
- ignored it
- deleted, w/o opening it

In one of the application that I built after quitting Oracle was Archive & Purge tool. This tool picks data from source, performs ETL (if user has defined) & inserts into the target, purges the records from the source. W/o getting too much into this tool (btw. I have 3 customers in various stages of implementation of this tool) I would like to highlight one major issue that I faced - building effective data slice that can be used as Payload for my workers.. I rummaged a lot, for guidelines on the same & happened to stumble upon an article by Tom Kyte.

This query gives you ranges of ROWID's that can be used based on chunks of data (num# of slices)

Here it goes......

http://asktom.oracle.com/pls/asktom/f?p=100:11:4288147952189703::::P11_QUESTION_ID:10498431232211

It took me a while to figure out what the query was actually doing.

However, I could not use this query in one of our implemtations as it just hung the system

- Query ran forever even after the entire schema was analyzed
- The customer had 35G of rollback - no issues with that
- Undo_retention of 36000
- We were trying to run this query on a table that had 127M records
- table size - 18G

I wrote a custom logic to build data slice on PK column after ensuring the table is analyzed & could get them up n running.

If any of you have a better way of building DS pl. free to share it. Any insights on why the query by Tom Kyte failed is also appreciated.

-Kiran.

No comments: