DOCSEARCHbyAlgolia
Submit Docs

Liuzaoqi's Technical Documentation

website·Indexed 31 days ago

https://liuzaoqi.com

A collection of technical notes and guides by an algorithm engineer covering machine learning model evaluation (PSI, KS, Confusion Matrix, WOE, IV), causal inference (Uplift models, PSM, ATE/ATT), data analysis with Pandas, and development tools including Docker and NVM.

Tokens
10.6K
Snippets
49
Records
81
Agent score
55%

What's inside Liuzaoqi

  1. Overview of Causal Inference Modeling Methods

    Causal inference aims to identify causal relationships by controlling variables to find identical samples. There are three core methodologies:

    1. Statistical Experiments: Using randomized controlled trials (e.g., A/B tests) to achieve perfect control variables.
    2. Quasi-experiments: Used when group assignment is non-random; requires manual sample adjustment to simulate control variables.
    3. Counterfactuals: Used when no control group exists; relies on machine learning models to generate synthetic counterfactual outcomes.

  2. Understand Propensity Score Matching (PSM) for Causal Inference

    Propensity Score Matching (PSM) is used to estimate the causal effect of an intervention by finding users in a non-treated group who are nearly identical to those in the treated group.

    Because matching on every individual feature (e.g., demographics, behavior) is computationally difficult due to high dimensionality, PSM uses a 'Propensity Score'—the predicted probability that a user belongs to the treatment group based on their characteristics. By matching users with similar propensity scores, researchers can create a homogeneous control group and eliminate confounding factors, allowing the difference in outcomes to be attributed to the intervention itself.

  3. Understand the necessity of Causal Inference over Correlation

    Causal inference is used to determine if a specific intervention (treatment) actually causes a specific result (outcome), rather than just being correlated. A common pitfall is ignoring 'confounding variables'—external factors that influence both the treatment and the outcome, leading to a false conclusion of causality.

    Common real-world scenarios requiring causal inference include:

    • Determining if a new recommendation strategy actually increases user retention or if the users were already high-retention users.
    • Assessing if a game advertisement increased login rates or if the targeted users were already high-potential users.

  4. Use Information Value (IV) to measure variable predictive power

    Information Value (IV) is a quantitative metric used to evaluate the predictive power of an independent variable relative to a binary target variable. It is particularly useful during the feature selection process to determine which variables should be included in a model. While WOE describes the relationship between a predictive variable and the target, IV measures the strength of that relationship. IV is preferred over raw WOE because it ensures the metric is non-negative and weights the contribution of each bin by its sample proportion relative to the total population.

  5. Understand the Potential Outcome Framework for Causal Inference

    The Potential Outcome Framework (also known as the Rubin Causal Model or RCM) defines causal effects by comparing what happens when an intervention is applied versus when it is not.

    Key concepts include:

    • Treatment (D): A binary indicator $D_i \in {0, 1}$ where $1$ indicates the user received the intervention (e.g., saw an ad) and $0$ indicates they did not.
    • Potential Outcomes ($Y_{1i}, Y_{0i}$): For every individual $i$, there are two potential outcomes: $Y_{1i}$ (the outcome if treated) and $Y_{0i}$ (the outcome if not treated).
    • Observed Outcome (Y): In reality, we only observe one outcome per person based on their treatment status: $Y_i = Y_{1i}$ if $D_i=1$, and $Y_i = Y_{0i}$ if $D_i=0$.
    • Counterfactual: The unobserved outcome. For a treated person, the counterfactual is $Y_{0i}$; for an untreated person, it is $Y_{1i}$.

  6. Understand Docker Images, Containers, and Volumes

    Docker relies on three core concepts to manage applications and data:

    1. Images: Read-only templates that serve as the foundation for containers. An image includes the operating system, software, applications, and libraries required to run a container. Images are reusable and used to instantiate one or more containers.
    2. Containers: Runnable instances created from images. Multiple containers can be started from a single image; each container is isolated, secure, and runs in its own environment.
    3. Volumes: Mechanisms for data persistence and sharing. Volumes exist independently of the container lifecycle and can be mounted from the host to a container or shared between containers to store and transfer data.

  7. Understand Uplift Modeling for Treatment Effect Estimation

    Uplift modeling is used to estimate the incremental impact (uplift) that a specific treatment action has on a user's response behavior. Unlike standard response models that predict the probability of an outcome, Uplift models predict the change in probability caused by the treatment. This allows businesses to segment users into four categories to optimize resource allocation:

    • Persuadables: Users who only perform the action if treated (High Uplift - Target these).
    • Sure Things: Users who perform the action regardless of treatment (Zero/Low Uplift - Waste of resources).
    • Lost Causes: Users who will not perform the action regardless of treatment (Zero/Low Uplift - Waste of resources).
    • Sleeping Dogs: Users who are deterred by the treatment and stop performing the action (Negative Uplift - Avoid these).

  8. Calculate Accuracy from a Confusion Matrix

    Accuracy measures the proportion of total correct predictions (both positive and negative) relative to the total number of samples. It provides a general overview of model performance but can be misleading for imbalanced datasets where one class dominates.
    Accuracy = (TP + TN) / (TP + TN + FP + FN)

  10. Use Information Value (IV) for variable selection

    Information Value (IV) is used to screen independent variables before fitting a classification model (such as Logistic Regression or Decision Trees). It helps developers select a subset of the most predictive variables from a large pool of candidates (e.g., selecting a few key variables out of 200) to form the final model input list.

  11. Enter a running or stopped Docker container

    Depending on the state of the container and the desired interaction, use different methods to enter the container:

    1. For stopped containers: First start the container using docker start, then use docker attach to connect to the main process.
    2. To connect to the main process: Use docker attach to join the session of the container's primary process.
    3. To start a new session: Use docker exec to run a new command (like a shell) inside an already running container. This is the preferred method for executing additional commands without disturbing the main process.
    # Start a stopped container
docker start my_python_app

# Attach to the main process
docker attach my_python_app

# Start a new bash shell session in a running container
docker exec -it my_python_app /bin/bash

  12. Use Causal Inference as an alternative to A/B Testing

    While A/B Testing is the most common method to find causal effects, it has several limitations that make causal inference from historical data necessary:

    • Resource Intensive: Requires significant engineering effort and time to implement.
    • Traffic Requirements: Requires a sufficient amount of randomized traffic and a duration of time to collect statistically significant data.
    • User Experience Risk: Some experiments may degrade the user experience (e.g., showing irrelevant content).
    • Scalability: When there are too many variations to test, it is impossible to run A/B tests for every option.

    Causal inference provides a framework to reasonably estimate potential outcomes using existing historical data without the need for active randomization.