Skip to main content

Posts

Showing posts from February, 2024

Module 4: Analytics Engineering (DTC DE Zoomcamp Week 4)

  dbt (Data Build Tool) Overview What is dbt? dbt stands for data build tool. It's a transformation tool that allows us to transform process raw data in our Data Warehouse to transformed data which can be later used by Business Intelligence tools and any other data consumers.   dbt  also allows us to introduce good software engineering practices by defining a deployment workflow:   1. Develop models 2. Test and document models 3. Deploy models with version control and CI/CD.   How does dbt work? dbt works by defining a modeling layer that stands on top of our DataWarehouse. Each table is turned into a model and then transformed into a derived model, that can be stored into the DataWarehouse for persistence.   A model consists in:   • *.sql file • Select statement, no DDL or DML are used • File that dbt will compile and run in our DataWarehouse   How to use dbt? dbt has 2 main components: dbt Core and dbt Cloud with the following characteristics:   dbt Cloud  - SaaS application to d