Archive for the ‘Extract Transform Load’ tag
What is ETL in Data Warehouse?
Data Warehouse
Data warehousing helps to provide information on the techniques involved in designing, building, maintaining and retrieving information, from a data warehouse. A data warehouse is premeditated and produced to support the decision-making process in an organization. The data that is obtained from the production databases are copied in the data warehouse, so that queries can be answered, without hindering the consistency of the production systems.
Data warehousing includes a set of important, new concepts and tools that have evolved into a technology. This makes it possible to counter the problems involved in providing all the key information, to the concerned people.
This field has evolved from the incorporation of a number of experiences and technologies, over the last two decades. Data warehousing is a well-organized and resourceful method of managing and reporting data from a variety of sources, non-uniform and scattered, throughout the company.
Data warehouses are vast, due to the hundreds of gigabytes of transactions. As a result, subsets, known as ‘data marts,’ are often designed for just one department or product line.
The data warehouse system serves as an influential and necessary platform, to merge the data from the old and new applications. Rules can also be transferred to the data warehouse, with very little effort. The most important and noteworthy features of a data warehouse is that it collects, records, filters and provides the basic data, to other organizational systems, at various higher levels.
Essentially, data warehouses are programmed to perform the summarization and pre-defining of the analysis, to automatically generate summary reviews. Data warehousing helps to analyze information for the users. This system is very useful for providing collective information to the users. Data warehousing systems have been created to support a variety of analysis, including elaborate queries on large amounts of data that require extensive searching.
What is ETL?
ETL is the process for Extracting, Transforming and Loading data from one database to another.
There are several ways for doing this, from coding your own processes to the more often used way of implementing ETL tools.
These ETL tools can do the job very well, and if chosen wisely can save you a lot of coding efforts and money, since you can graphically build processes and in most cases without knowing how to program for databases.
There are a lot of ETL tools in the market right now. As and advice, I suggest you to invest some research time (and testing if possible) before choosing the one that suits your company’s needs.
They can range from open source free tools to high price commercial tools. Neither of them is perfect in every situation, and you will have to take into account your data volumes, the analysis and answers you want from your datawarehouse, and the periodicity needed of those answers, among other aspects.