Home
Disclaimer
Required Tools
Setup Workspace
1.
Big Data Overview
1.1.
Introduction
1.2.
Job Opportunities
1.3.
What is Data?
1.4.
How does it help?
1.5.
Types of Data
1.6.
The Big V's
1.6.1.
Variety
1.6.2.
Volume
1.6.3.
Velocity
1.6.4.
Veracity
1.6.5.
Other V's
1.7.
Trending Technologies
1.8.
Big Data Concerns
1.9.
Big Data Challenges
1.10.
Data Integration
1.11.
Scaling
1.12.
CAP Theorem
1.13.
PACELC Theorem
1.14.
Optimistic Concurrency
1.15.
Eventual Consistency
1.16.
Concurrent vs Parallel
1.17.
GPL
1.18.
DSL
1.19.
Big Data Tools
1.20.
NO Sql Databases
1.21.
Learning Big Data means?
2.
Developer Tools
2.1.
Introduction
2.2.
UV
2.3.
Other Python Tools
3.
Data Format
3.1.
Introduction
3.2.
CSV-TSV
3.3.
JSON
3.4.
Parquet
3.5.
Arrow
3.6.
Avro
3.7.
YAML
3.8.
Duck DB
4.
Protocol
4.1.
Introduction
4.2.
HTTP
4.3.
Monolithic Architecture
4.4.
Statefulness
4.5.
Microservices
4.6.
Statelessness
4.7.
Idempotency
4.8.
REST API
4.9.
API Performance
4.10.
API in Big Data world
5.
Advanced Python
5.1.
Data Frames
5.2.
Decorator
5.3.
Unit Testing
5.4.
Error Handling
5.5.
Logging
6.
Containers
6.1.
CPU Architecture Fundamentals
6.2.
Introduction
6.3.
VMs or Containers
6.4.
What Container does
6.5.
Docker
6.6.
Docker Examples
7.
CICD
7.1.
Introduction
7.2.
CICD Tools
7.3.
CI Yaml
7.4.
CD Yaml
8.
Data Engineering
8.1.
Introduction
8.2.
Batch vs Streaming
8.3.
Medallion Architecture
8.4.
Data Quality Checks
8.5.
Data Engineering Model
8.6.
Quality & Governance
8.7.
Data Mesh
8.8.
KAFKA
8.8.1.
Introduction
8.8.2.
Use Cases
8.8.3.
Kafka Software
8.8.4.
Python Scripts
8.8.5.
Different types of streaming
9.
Cloud Computing
9.1.
Introduction
9.2.
Types of Cloud Services
9.3.
Challenges
9.4.
Multi-Cloud
9.5.
High Availability
9.6.
Disaster Recovery (DR)
9.7.
RTO - RPO
9.8.
Cloud Native vs Lift-and-Shift
9.9.
Azure Cloud
9.9.1.
Services
9.9.2.
Azure Access
9.9.3.
IaC
9.9.4.
Idempotency
9.9.5.
Terraform
10.
CLI Tools
10.1.
Introduction
10.2.
Linux Commands 01
10.3.
Linux Commands 02
10.4.
AWK
10.5.
CSV SQL
10.6.
JQ
10.7.
YQ
11.
Miscellaneous
11.1.
Additional Reading
11.2.
Good Reads
11.3.
Roadmap Data Engineer
11.4.
Notebooks vs IDE
Tags
Light
Rust
Coal
Navy
Ayu
Big Data Tools & Techniques
[Avg. reading time: 1 minute]
The Big V’s of Big Data
Variety
Volume
Velocity
Veracity
Other V’s
#bigv
#bigdata
Ver 6.0.25