Skip to content

Latest commit

 

History

History

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 

README.md

🛠️ Skills & Knowledge

Python for Data Engineering (Practice)

  • Data Cleansing: Using .strip(), .upper(), and .split() to clean and format malformed string data, and f-strings for reporting.

  • Error Handling & Type Casting: Implementing try-except-finally blocks to catch exceptions, handling specific workflows like ZeroDivisionError, and using float() for safe data conversion.

  • Object-Oriented Programming (OOP) for Pipelines: Implementing class structures using @dataclass, initializing internal states, and defining post-initialization hooks via __post_init__ for automated data standardization.

  • Dynamic Data Ingestion (*args): Using dynamic positional arguments to handle and unpack variable numbers of data batches seamlessly within methods.

  • List & Dict Comprehensions: Utilizing List Comprehensions and for loops with if-else conditionals to filter data based on business rules, and Dictionary Comprehensions for transforming structured data formats (Key-Value mappings).

  • Data Structures:

  • Lists: Creating via list(), adding elements with .append() / .extend(), and duplicating states using .copy().

  • Dictionaries: Initializing empty dictionaries {} and mapping structured key-value pairs.

  • Sets: Initializing with set() and using .add() to eliminate duplicates (Deduplication).

  • Built-in Functions: Using round() for decimal formatting, sum() and len() for aggregations, and enumerate() for custom loop indexing.

  • Memory Optimization: Applying .clear() to wipe out datasets and optimize memory usage after processing.


📂 Project Structure

1. python-data-cleaning-practice.py

  • Content: A 10-step Python data processing workflow.
  • Focus: Hands-on practice with core Python data structures, error safety, and list comprehension techniques.

2. Python-OOP.py

  • Content: Designing a reliable sales data pipeline with Python classes and exception handling..
  • Focus: Harnessing the power of Python @dataclass, dynamic argument packing (*args), dictionary transformations via comprehensions, and resilient try-except error management to process multi-batch transactions safely.