How I Slashed a 1 million Email processing Pipeline from 11 Days to 38 Hours with Lightweight Parallelism

In the era of Generative AI, the quality and scale of data processing have become more critical than ever. While sophisticated language models and ML algorithms steal the spotlight, the behind-the-scenes work of data preparation remains the unsung hero of successful AI implementations. From cleaning inconsistent formats to transforming raw inputs into structured information, these preparatory steps directly impact model performance and output quality. However, as data volumes grow exponentially, traditional sequential processing approaches quickly become bottlenecks, turning what should be one-time tasks into resource-intensive operations that delay model training and deployment. For organizations working with moderate to large datasets—too small to justify a full Hadoop or Spark implementation, yet too unwieldy for single-threaded processing—finding the middle ground of efficient parallelism has become essential for maintaining agile AI development cycles. ...

April 18, 2025 · 16 min

Custom Validations in Django Rest Framework

Introduction One of the key components of HTTP request-response is request data validation. Modern HTTP requests typically send a JSON payload that needs to be sanitized and validated before proceeding with business logic and database operations. While client-side applications may implement basic form validations using libraries like Formik, backend validation remains essential as a security precaution. This article on hacking McDonald’s India Service APIs showcases interesting vulnerabilities that can be exploited when backend validations and authorizations are improperly implemented. ...

March 21, 2025 · 7 min

Hacky usage of mutable default arguments in Python

One of the most commonly known gotchas in Python is the use of mutable default arguments. Consider this simple Python function snippet: def foo(item: int, bar: list = []) -> None: bar.append(item) print( f"{bar=}" ) # Neat f-string trick btw to print both variable name and value foo(6) foo(6) foo(12) A Python newcomer might expect the output to be: bar = [6] bar = [6] bar = [12] Instead you would get: ...

February 12, 2025 · 5 min