Advanced Python Concepts and Best Practices
Open in the Padhaiko app1.1 Review of Python essentials and coding conventions
Writing maintainable Python code for data science requires mastering core language semantics alongside clean formatting and documentation standards. Data pipelines are often read, debugged, and extended by multiple engineers, so consistency directly reduces bugs and onboarding time.
Core Python Essentials
- Dynamic Typing and Object References: Variables in Python are names bound to objects in memory, not fixed-size storage boxes. Every object carries a type, an identity (
id()), and a value. - Mutability vs. Immutability:
- Immutable types (
int,float,str,tuple,frozenset,bytes) cannot be altered after creation; operations return new objects. - Mutable types (
list,dict,set,bytearray) can be modified in place. Passing a mutable object to a function allows the function to mutate the caller's object unless a copy is made.
- Immutable types (
- Truthiness and Identity: Empty containers (
[],{},set(),""),0,0.0, andNoneevaluate toFalsein boolean contexts. Useis/is notonly for identity checks (especiallyx is None), and==/!=for value equality. - Comprehensions and Unpacking: List, dictionary, and set comprehensions replace verbose loops with concise expressions, while sequence unpacking (
first, *middle, last = seq) cleanly extracts elements.
| Aspect | Mutable Objects | Immutable Objects |
|---|---|---|
| In-place Modification | Allowed (append, item assignment) | Raises TypeError; creates a new object instead |
| Hashability | Unhashable by default (cannot be dict keys) | Hashable if all nested elements are immutable |
| Default Argument Trap | Unsafe as function defaults (def f(x=[]):) | Safe as function defaults (def f(x=None):) |
| Thread / Pipeline Safety | Requires defensive copying (.copy() / deepcopy) | Inherently safe to share across functions |
Coding Conventions (PEP 8, Type Hints, and Docstrings)
PEP 8 is the official style guide for Python code. Adhering to PEP 8 keeps scientific notebooks, reusable modules, and production packages uniform and readable.
PEP 8, Type Hints, and Docstring Standards
- Indentation and Line Length: Use 4 spaces per indentation level (never mix tabs and spaces). Limit source lines to 79 characters (72 for docstrings/comments) or the project-agreed 88 characters.
- Naming Conventions:
snake_casefor functions, methods, variables, and module names (e.g.,compute_zscore,data_loader.py).PascalCase(CapWords) for classes and custom exceptions (e.g.,StandardScaler,MissingColumnError).UPPER_SNAKE_CASEfor module-level constants (e.g.,MAX_RETRIES = 3,DEFAULT_ALPHA = 0.05).
- Imports Order: Group imports at the top of the file in three blocks separated by blank lines: (1) standard library (
os,math,json), (2) third-party packages (numpy,pandas), and (3) local application modules. - Type Annotations (PEP 484): Annotate function parameters and return types (e.g.,
def normalize(vals: list[float]) -> list[float]:) to enable static analysis (mypy,pyright) and IDE autocompletion. - Docstrings (PEP 257): Document modules, classes, and public functions with triple-quoted strings (
"""...""") summarizing purpose, parameters, return values, and raised exceptions (NumPy or Google docstring style is standard in data science).
from typing import Iterable
def mean_center(values: Iterable[float]) -> list[float]:
"""Subtract the arithmetic mean from each numeric value.
Args:
values: An iterable of numeric observations.
Returns:
A list of centered floating-point values.
Raises:
ValueError: If values is empty.
"""
data = list(values)
if not data:
raise ValueError("Cannot mean-center an empty sequence.")
avg = sum(data) / len(data)
return [x - avg for x in data]1.2 Advanced data structures: Collections, iterators, generators, and decorators
Standard lists and dictionaries handle general tasks, but large-scale data processing demands specialized containers, lazy iteration, and clean cross-cutting wrappers.
The collections Module
Python's built-in collections module supplies specialized container datatypes tuned for frequency counting, grouping, sliding windows, and lightweight records:
| Class | Underlying Idea | Primary Data Science Use Case |
|---|---|---|
Counter | dict subclass mapping hashable items to counts | Word frequencies, class label counts, .most_common(k) |
defaultdict | dict with automatic factory for missing keys | Grouping records by category without manual key checks |
deque | Doubly-linked list with O(1) ends | Sliding-window moving averages (deque(maxlen=k)), BFS queues |
namedtuple | Immutable tuple with named attribute fields | Lightweight read-only data rows without class boilerplate |
ChainMap | Layered view over multiple dictionaries | Merging CLI flags, environment settings, and default configs |
from collections import Counter, defaultdict, deque, namedtuple
# 1. Frequency counting
labels = ["spam", "ham", "ham", "spam", "ham", "ham"]
counts = Counter(labels) # Counter({'ham': 4, 'spam': 2})
# 2. Grouping by key
groups = defaultdict(list)
for dept, score in [("CS", 88), ("ECE", 76), ("CS", 92)]:
groups[dept].append(score) # {'CS': [88, 92], 'ECE': [76]}
# 3. Fixed-size sliding window
window = deque(maxlen=3)
for val in [10, 20, 30, 40]:
window.append(val) # ends as deque([20, 30, 40], maxlen=3)
# 4. Lightweight record
Record = namedtuple("Record", ["id", "feature", "label"])
row = Record(101, 0.85, 1)Iterators and the Iterator Protocol
Iterable vs. Iterator Protocol
- An iterable is any object capable of returning its members one at a time; it implements
__iter__()returning an iterator (examples:list,tuple,dict,str,range). - An iterator is an object representing a stream of data; it implements both:
__iter__(self)returningself, and__next__(self)returning the next element or raisingStopIterationwhen exhausted.
- Calling
iter(obj)invokesobj.__iter__(), and callingnext(it)invokesit.__next__(). An iterator is stateful and single-pass: once exhausted, it continues raisingStopIteration.
Generators (yield and Generator Expressions)
A generator is a concise way to build an iterator without writing __iter__ and __next__ manually.
- Generator function: Any function containing
yieldreturns a generator object when called. Execution pauses at eachyield, preserving local variables and instruction pointer until the nextnext()call. - Generator expression: Uses parentheses
(expr for x in iterable if cond)to produce values lazily one at a time, consuming O(1) memory regardless of sequence size—crucial when streaming multi-gigabyte log or CSV files.
| Aspect | List Comprehension | Generator Expression |
|---|---|---|
| Evaluation Strategy | Eager (computes all items immediately) | Lazy (computes one item on demand) |
| Memory Usage | O(N) — stores entire list in RAM | O(1) — holds only current state frame |
| Reusability | Reusable; supports indexing lst[i] and len() | Single-pass exhaustion; no indexing or len() |
| Best Used For | Small/medium data needing random access or sorting | Large streams, file pipelines, sum() / any() reductions |
Decorators
A decorator is a higher-order callable that takes a function (or class), wraps it with additional behavior, and returns a new callable—without modifying the wrapped function's source code. Decorators rely on Python's first-class functions and closures, and use @functools.wraps(func) so the wrapper preserves __name__, __doc__, and type metadata.
Common data science uses include execution timing, result memoization (@functools.lru_cache), input validation, and retry logic for flaky API requests.
import functools
import time
def timer(func):
"""Measure and print execution time of the decorated function."""
@functools.wraps(func)
def wrapper(*args, **kwargs):
t0 = time.perf_counter()
result = func(*args, **kwargs)
elapsed_ms = (time.perf_counter() - t0) * 1000.0
print(f"{func.__name__} took {elapsed_ms:.2f} ms")
return result
return wrapper
@timer
def sum_squares(n: int) -> int:
return sum(i * i for i in range(1, n + 1))1.3 Functions and lambda expressions
Functions decompose complex analytical workflows into testable, reusable units. Python supports rich parameter-passing semantics alongside compact anonymous functions for functional transformations.
Function Arguments and Scope (LEGB Rule)
- Argument Kinds:
- Positional and Keyword arguments:
func(10, scale=2.0). - Default arguments:
def clip(x, low=0.0, high=1.0):. - Arbitrary positional (
*args): Collects extra positional arguments into atuple. - Arbitrary keyword (
**kwargs): Collects extra keyword arguments into adict. - Positional-only (
/) and Keyword-only (*): Enforced in APIs likedef train(X, y, /, *, epochs=10, lr=0.01):so callers cannot pass hyperparameters positionally by accident.
- Positional and Keyword arguments:
- Variable Scope (LEGB): Python resolves unqualified names in order: Local → Enclosing (nested functions / closures) → Global (module) → Built-in (
len,sum). Usenonlocalto rebind an enclosing variable and avoidglobalin data pipelines.
Lambda Expressions and Functional Primitives
A lambda expression defines a small, single-expression anonymous function:
lambda arguments: single_expression
Lambdas cannot contain statements (return, for, try, assignments) and return the value of single_expression automatically. They are most idiomatic as short inline callbacks (key= functions) and in functional pipelines (map, filter, functools.reduce):
| Function | Signature / Behavior | Typical Example |
|---|---|---|
sorted(seq, key=...) | Returns new sorted list using key function | sorted(rows, key=lambda r: r["score"], reverse=True) |
map(fn, iter) | Lazy iterator applying fn to each item | list(map(lambda x: x / 100.0, [25, 50, 75])) |
filter(pred, iter) | Lazy iterator keeping items where pred(x) is truthy | list(filter(lambda x: x > 0, [-3, 0, 4, 9])) |
functools.reduce(fn, iter) | Folds items left-to-right into one cumulative value | reduce(lambda a, b: a * b, [2, 3, 4]) → 24 |
1.4 Object-Oriented Programming for data science applications
Modern Python ML and data libraries (scikit-learn, PyTorch, pandas) organize datasets, preprocessors, and estimators as classes. Understanding OOP enables you to write custom transformers, models, and modular data containers.
Classes, Instances, and Special (Dunder) Methods
Core OOP Concepts in Data Science
- Encapsulation: Bundling data attributes (e.g., learned
mean_andstd_fitted on training data) and methods (fit,transform) inside a single class prevents data leakage and keeps pipeline state organized. - Inheritance and Polymorphism: Defining a shared base interface (such as
BaseEstimatorwith.fit()and.predict()) allows a pipeline runner to call any model polymorphically regardless of internal algorithm. - Abstract Base Classes (
abc.ABC): Use@abstractmethodto enforce that subclasses implement mandatory methods before instantiation. - Dunder (Magic) Methods:
__init__: initializes instance attributes.__repr__: unambiguous developer representation (shown in notebooks/logs).__len__and__getitem__: let custom dataset classes supportlen(ds)and indexingds[i].__call__: makes an instance callable like a function (used in custom transforms and neural layers).
- Dataclasses (
@dataclass): Automatically generate__init__,__repr__, and__eq__for configuration and record classes, cutting boilerplate.
from abc import ABC, abstractmethod
from dataclasses import dataclass
@dataclass(frozen=True)
class ScalerConfig:
with_mean: bool = True
eps: float = 1e-8
class BaseTransformer(ABC):
@abstractmethod
def fit(self, data: list[float]) -> "BaseTransformer":
pass
@abstractmethod
def transform(self, data: list[float]) -> list[float]:
pass
def fit_transform(self, data: list[float]) -> list[float]:
return self.fit(data).transform(data)
class ZScoreScaler(BaseTransformer):
def __init__(self, config: ScalerConfig | None = None):
self.config = config or ScalerConfig()
self.mean_: float = 0.0
self.std_: float = 1.0
def fit(self, data: list[float]) -> "ZScoreScaler":
if not data:
raise ValueError("Cannot fit ZScoreScaler on empty data.")
self.mean_ = sum(data) / len(data)
var = sum((x - self.mean_) ** 2 for x in data) / len(data)
self.std_ = (var ** 0.5) or self.config.eps
return self
def transform(self, data: list[float]) -> list[float]:
shift = self.mean_ if self.config.with_mean else 0.0
return [(x - shift) / self.std_ for x in data]
def __repr__(self) -> str:
return f"ZScoreScaler(mean_={self.mean_:.3f}, std_={self.std_:.3f})"Notice the scikit-learn convention: attributes learned during .fit() carry a trailing underscore (mean_, std_), and .fit() returns self to allow method chaining (scaler.fit(X_train).transform(X_test)).
1.5 Exception handling, debugging, and logging
Unattended data ingestion and training jobs encounter corrupt rows, network timeouts, missing keys, and division by zero. Graceful error handling, structured logging, and systematic debugging separate brittle scripts from production pipelines.
Exception Handling (try / except / else / finally)
Exception Handling Flow and Custom Exceptions
try: wraps code that may raise an exception.except SpecificError as err: catches only expected exception types (never use bareexcept:, which hidesKeyboardInterruptand typos).else: runs only if thetryblock succeeded with no exception—keeps normal follow-up code out of the guarded block.finally: always runs (whether an exception occurred or not), used for cleanup such as closing file handles or database connections (often paired withwithcontext managers).raise: re-raises an exception or raises a domain-specific custom exception inheriting fromException.
class SchemaValidationError(ValueError):
"""Raised when an input record violates expected column schema."""
def parse_positive_float(raw: str) -> float:
try:
val = float(raw)
except ValueError as exc:
raise SchemaValidationError(f"Non-numeric value: {raw!r}") from exc
if val <= 0:
raise SchemaValidationError(f"Expected positive value, got {val}")
return valDebugging Techniques
- Traceback Reading: Read Python tracebacks bottom-up: the last line gives the exception type and message, while the stack frame just above it points to the failing line in your code.
- Interactive Debugger (
pdb/breakpoint()): Insertingbreakpoint()(Python 3.7+) pauses execution at that line and openspdb, where commandsn(next line),s(step into),c(continue),p expr(print variable), andl(list source) inspect live state. - Assertions (
assert): Useassert len(X) == len(y), "Mismatched samples"for internal sanity checks during development—never for validating untrusted user input, becausepython -Ostripsassertstatements.
Structured Logging with logging
Replacing print() calls with Python's standard logging module provides severity filtering, timestamps, module names, and simultaneous output to console and rotating log files:
| Level | Numeric Value | When to Use in Data Science Pipelines |
|---|---|---|
DEBUG | 10 | Detailed shapes, intermediate tensors, row-level diagnostics |
INFO | 20 | Pipeline milestones (rows loaded, epoch accuracy, job complete) |
WARNING | 30 | Recoverable anomalies (missing optional column, fallback used) |
ERROR | 40 | Failed record batch, skipped file, API call failure |
CRITICAL | 50 | Fatal configuration or storage failure halting the entire pipeline |
import logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
)
logger = logging.getLogger("etl.cleaner")
logger.info("Loaded %d valid rows from batch %s", 1250, "2026-11-18")1.6 Working with modules and packages
Modularizing code into modules and packages prevents monolithic notebooks, enables unit testing, and supports reproducible environments across machines.
Modules, Packages, and Import Resolution
- Module: Any single
.pyfile containing Python definitions and statements (e.g.,metrics.py). - Package: A directory containing Python modules and typically an
__init__.pyfile (regular package) that exposes a clean public API and organizes submodules (dataprep/cleaning.py,dataprep/scaling.py). __name__ == "__main__"Guard: When Python imports a module,__name__equals the module's dotted name ("dataprep.cleaning"); when executed directly (python cleaning.py),__name__is"__main__". Wrapping CLI/test code insideif __name__ == "__main__":prevents side effects during imports.- Module Search Path (
sys.path): Python searches (1) the script's directory (or current working directory), (2) directories inPYTHONPATH, and (3) standard library +site-packages. - Virtual Environments (
venv): Isolate project dependencies (python -m venv .venv,pip install -r requirements.txtorpyproject.toml) so conflicting library versions across projects do not break pipelines.
my_ds_project/
|-- pyproject.toml # Build metadata and pinned dependencies
|-- README.md
|-- src/
| `-- pipeline/
| |-- __init__.py # Re-exports public classes/functions
| |-- ingest.py # Data loading module
| |-- transform.py # Feature scaling & cleaning
| `-- models/
| |-- __init__.py
| `-- trainer.py # Subpackage module
`-- tests/
`-- test_transform.py # Unit tests for transform moduleUse absolute imports (from pipeline.transform import ZScoreScaler) across packages for clarity, and keep relative imports (from .ingest import load_csv) strictly within the same package.
Worked Examples
1.1
Write a Python pipeline using a generator expression and a custom decorator to compute the sum of squares of even numbers in [1, 2, 3, 4, 5, 6], and trace the exact output and memory behavior. Also trace the output of a deque(maxlen=2) sliding average on [5, 10, 15].
Building the Decorator and Generator Pipeline: Define a @log_call decorator using functools.wraps to preserve metadata, and pass a lazy generator expression (x ** 2 for x in nums if x % 2 == 0) to sum():
import functools
def log_call(fn):
@functools.wraps(fn)
def wrapper(*args, **kwargs):
res = fn(*args, **kwargs)
print(f"{fn.__name__} -> {res}")
return res
return wrapper
@log_call
def even_square_sum(nums: list[int]) -> int:
return sum(x ** 2 for x in nums if x % 2 == 0)Tracing Execution and Sliding Window:
- Filtered even numbers from
[1, 2, 3, 4, 5, 6]are2, 4, 6. - Generator yields
2**2 = 4, then4**2 = 16, then6**2 = 36one at a time in O(1) auxiliary space. sum(...)accumulates4 + 16 + 36 = 56, and@log_callprintseven_square_sum -> 56.- For
window = deque(maxlen=2)over[5, 10, 15]:- After
5:[5](avg =5.0) - After
10:[5, 10](avg =(5 + 10) / 2 = 7.5) - After
15:5is evicted automatically →[10, 15](avg =(10 + 15) / 2 = 12.5)
- After
even_square_sum([1, 2, 3, 4, 5, 6]) prints even_square_sum -> 56 and returns 56 in O(1) extra memory. The deque(maxlen=2) sliding averages across [5, 10, 15] are 5.0, 7.5, and 12.5.
1.2
Implement a MinMaxNormalizer class with fit() and transform() methods following scikit-learn OOP conventions, and compute the normalized output for training data [10, 20, 30] and test value [25].
Implementing the OOP Transformer: Store min_ and max_ as trailing-underscore attributes fitted only on training data, guard against zero range (max_ == min_), and handle unseen test data with the stored bounds:
class MinMaxNormalizer:
def __init__(self) -> None:
self.min_: float = 0.0
self.max_: float = 1.0
def fit(self, data: list[float]) -> "MinMaxNormalizer":
if not data:
raise ValueError("Training data cannot be empty.")
self.min_ = min(data)
self.max_ = max(data)
return self
def transform(self, data: list[float]) -> list[float]:
span = self.max_ - self.min_
if span == 0:
return [0.0 for _ in data]
return [(x - self.min_) / span for x in data]Evaluating on [10, 20, 30] and Test [25]:
- Fitting on
[10, 20, 30]setsmin_ = 10,max_ = 30, sospan = 30 - 10 = 20. - Transforming training data:
(10 - 10) / 20 = 0.0(20 - 10) / 20 = 0.5(30 - 10) / 20 = 1.0
- Transforming test value
[25]using the trainingmin_andmax_prevents data leakage:(25 - 10) / 20 = 0.75.
fit([10, 20, 30]) sets min_ = 10 and max_ = 30. transform([10, 20, 30]) returns [0.0, 0.5, 1.0], and transform([25]) returns [0.75].
1.3
How should a student answer the exam question: "Explain Python's iterator protocol, contrast iterators with generators, and show how try / except / else / finally and logging work together in a resilient data loader"?
Structuring the Protocol & Comparison:
- State the iterator protocol:
__iter__()returns the iterator object (self), and__next__()returns the next value or raisesStopIteration. - Contrast class-based iterators (explicit state class with
__iter__+__next__) with generators (functions usingyieldor(expr for x in seq)that compile the state machine automatically and pause between yields). - Include a 3-line code snippet showing
next()andStopIterationoryield.
Integrating Exception Handling & Logging: Write a concise loader pattern showing all four try / except / else / finally clauses and logging levels (INFO on success, ERROR on missing/corrupt file, finally for guaranteed cleanup), and explain why bare except: must be avoided.
A full-mark answer defines __iter__() and __next__() (StopIteration), contrasts manual iterator classes against yield generators in memory and syntax, and illustrates a try / except FileNotFoundError / else / finally file loader paired with logging.getLogger() at INFO/ERROR levels.
Exercises
Practice Exercises — Advanced Python Concepts and Best Practices
Explain PEP 8 naming conventions, type hints, and the mutable default argument trap in Python. Show how to fix a buggy function def add_item(item, bucket=[]):.
PEP 8 Conventions and Type Hints:
- Functions, methods, and variables use
snake_case; classes usePascalCase; constants useUPPER_SNAKE_CASE; indentation uses 4 spaces per level. - Type hints (PEP 484) annotate argument and return types (e.g.,
def add_item(item: str, bucket: list[str] | None = None) -> list[str]:) for static verification and clearer APIs.
Fixing the Mutable Default Argument Trap: Default expressions are evaluated once when def executes, so bucket=[] creates a single shared list across every call omitting bucket. Replace [] with None and allocate inside the body:
def add_item(item: str, bucket: list[str] | None = None) -> list[str]:
if bucket is None:
bucket = []
bucket.append(item)
return bucketPEP 8 standardizes snake_case functions/variables, PascalCase classes, UPPER_SNAKE_CASE constants, 4-space indentation, and grouped imports; type hints specify expected types statically. Because default arguments are evaluated once at function definition time, replace bucket=[] with bucket=None and initialize bucket = [] inside the function body.
Given a list of product records [("pen", 20), ("book", 100), ("bag", 80)], write Python code using lambda with sorted() and filter() to select items costing at least 50, sort them by price descending, and compute a 25% discount on the highest-priced item.
Filtering and Sorting with Lambda:
items = [("pen", 20), ("book", 100), ("bag", 80)]
expensive = list(filter(lambda item: item[1] >= 50, items))
ranked = sorted(expensive, key=lambda item: item[1], reverse=True)filterkeeps("book", 100)and("bag", 80)("pen"at 20 is dropped).sorted(..., reverse=True)orders them as[("book", 100), ("bag", 80)].
Computing the 25% Discount on the Top Item: The highest-priced item is ranked[0] = ("book", 100).
- Discount amount =
100 * 0.25 = 25.0. - Discounted price =
100 - 25.0 = 75.0.
Filtered and sorted list: [("book", 100), ("bag", 80)]. A 25% discount on "book" (100) subtracts 25.0, giving a discounted price of 75.0.
Differentiate between an iterator and a generator in Python. Write a generator function squares_up_to(n) that yields 1^2, 2^2, ..., n^2, and calculate the sum yielded for n = 5.
Iterator vs. Generator:
- An iterator is any object implementing
__iter__()and__next__(), often requiring boilerplate state attributes in a custom class. - A generator is a special iterator created either by a function using
yieldor by a generator expression(expr for x in seq). Python automatically generates__iter__(),__next__(), state preservation, andStopIterationon return.
Generator Implementation and Trace for n = 5:
def squares_up_to(n: int):
for i in range(1, n + 1):
yield i * iYielded values for n = 5: 1, 4, 9, 16, 25. Total sum: 1 + 4 + 9 + 16 + 25 = 55.
Every generator is an iterator, but generators automate __iter__/__next__ state management via yield (or (expr for ...)). For n = 5, squares_up_to(5) yields 1, 4, 9, 16, 25, whose sum is 55.
What is a decorator in Python, and why is @functools.wraps(func) important when writing one? Write a @require_non_empty decorator that raises ValueError if the first positional argument is empty.
Decorator Definition and functools.wraps: A decorator is a callable that accepts a function, wraps it in an inner closure adding pre/post behavior, and returns the wrapper. Without @functools.wraps(func), the wrapper overwrites func.__name__, func.__doc__, and __wrapped__, breaking introspection, logging, and documentation tools.
Implementing @require_non_empty:
import functools
def require_non_empty(func):
@functools.wraps(func)
def wrapper(seq, *args, **kwargs):
if len(seq) == 0:
raise ValueError(f"{func.__name__} requires a non-empty sequence.")
return func(seq, *args, **kwargs)
return wrapperA decorator wraps a callable to extend its behavior without modifying its source code, and @functools.wraps(func) preserves the original function's __name__, __doc__, and signature metadata. The @require_non_empty wrapper checks len(seq) == 0, raises ValueError, and otherwise forwards arguments to func.
Explain how custom exceptions, try / except / else / finally, logging, and Python packages (__init__.py, __name__ == "__main__") work together to make a data science project production-ready.
Resilient Execution with Exceptions and Logging:
- Custom exceptions (inheriting from
ExceptionorValueError) give callers meaningful domain errors (e.g.,MissingColumnError) instead of generic failures. try / except / else / finallyisolates risky I/O, catches specific exceptions, runs normal post-load logic inelse, and guarantees resource release infinally.loggingrecords timestampedINFO,WARNING, andERRORevents to files and consoles without relying on unfilterableprint()calls.
Modular Structure with Packages and Guards:
- Grouping code into a package (
__init__.py+ focused submodules likeingest.py,features.py) enables clean imports and isolated unit testing. - Placing script entrypoints behind
if __name__ == "__main__":allows modules to be imported safely by test suites and downstream pipelines without triggering execution side effects.
Custom exceptions provide domain-specific error semantics; try / except / else / finally guarantees safe handling and cleanup; logging records severity-tagged audit trails; and packages (__init__.py) combined with if __name__ == "__main__": separate reusable library code from runnable scripts.