Cross-entropy is a good perspective to understand logistic regression, but I have the following question:
the objective function of LR: $$\max L(\theta) = \max \sum_{i=1}^N y_i \log \hat y_i + (1-y_i) \log (1- \hat y_i)$$ where $y_i$ is the probability of true label，$\hat y_i$ is the probability of predicted label. Now if we have $p \in \{y,1-y\}$ and $q \in \{\hat{y}, 1-\hat{y}\}$,then the cross-entropy of $p,q$ has the following two formulations: $$H_p(q)=\sum_x q\log \frac{1}{p}$$ and $$H_q(p)=\sum_x p\log \frac{1}{q}$$ According to the information theory, we know that these two formulations are not symmetry, that is they are not equal to each other. Usually, we may use the second formulation to understand the logistic regression and my question is that why not use the first one?
In my opinion, the first one may be more suitable for the thought of LR. From the view of information theory, since $p$ is the baseline distribution, we may use the optimal code of $p$ ($\log\frac{1}{p}$ represents the optimal code) to value the average code for distribution $q$, then we may try to minimize $H_p(q)$ in order to make $q$ more suitable for the optimal code. Then for LR, we are also looking forward to have a distribution $q$ which is similar to the true distribution $p$. I'm not sure if I have made a mistake.