Master of Science in Data Science @NEU
Blogs - My technical summary and daily sharing
Notes - Learning Notes
Always happy to chat, whether it’s serious discussion or casual conversation!
GitHub: @YoungZ2357
Student email：zhang.q4@northeastern.edu
Personal email：zqy2357@outlook.com
IMPORTANT: This note is translated by LLM and fixed manually. See Chinese version for a more accurate note 1. Method of Lagrange Multipliers 1.1 Basic Definition Lagrange first proposed the Method of Lagrange Multipliers for equality constraints. Consider the following optimization problem: $$ \begin{aligned} & \text{minimize} \quad & f_0(x)& \\\\ & \text{subject to} \quad & h_i(x) & = 0, \quad i=1, \cdots, m \end{aligned} $$Then we have the Lagrangian function: ...
IMPORTANT: This note is translated by LLM and fixed manually. See Chinese version for a more accurate note 1. General Descent Methods 1.1 Basic Form of Descent Methods Note: Descent methods do not require convexity, but convexity provides significant guarantees for solving optimization problems. The prototype of gradient descent is the descent method. A descent algorithm generates a sequence of optimization points $x^{(k)}, k=1, \cdots$, where $$ x^{(k+1)} = x^{(k)} + t^{(k)}\Delta x^{(k)} $$and $t^{(k)} > 0$ (unless $x^{(k)}$ is already optimal). ...
IMPORTANT: This note is translated by LLM and fixed manually. See Chinese version for a more accurate note 1. Algorithm Overview Logistic regression predicts the probability of an event by mapping the output of linear regression to the probability space: $$ P(y=1|x;\theta)=h_{\theta}(x)=\frac{1}{1+e^{-\theta^Tx}} $$ This formula represents: given model parameters $\theta$ and input feature values $x$, the probability that the classification label $y=1$ is $h_{\theta}(x)$. The pseudocode for a single inference of this model is approximately as follows: ...