database - Best practices for huge volumes of data load/unload? -
database - Best practices for huge volumes of data load/unload? - my question applies etl scenarios, transformation performed outside of database (completely). if extract, transform, , load huge volumes of info (20+ 1000000 records or more) , databases involved : oracle , mssql server, best way to: effectively read source database : there way avoid querying on network? have heard things direct path extract method/ mass unload method - i'm quite not sure how work, presume need dump file of sorts kind of non-network based info read/import? effectively write transformed info target database?: should consider apache hadoop? help me start transformation , parallely load info destination database? - faster say, oracle's mass load utility? if not,, there way remote invoke mass load utlities on oracle/mssql server? appreciate thoughts/suggestions. i utilize db's mass load facilities this. remote command of mass loads sysadmin issue; there way this....